Apr 13 02:26:21.887685 np0000165409 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]: we strongly recommend against using it for new projects.
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]: If you are already using Eventlet, we recommend migrating to a different
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]: framework.  For more detail see
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
Apr 13 02:26:22.207957 np0000165409 neutron-ovn-maintenance-worker[106236]:   import eventlet
Apr 13 02:26:23.328577 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.common.config [-] Logging enabled!
Apr 13 02:26:23.328577 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev111
Apr 13 02:26:23.329332 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=106236) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
Apr 13 02:26:23.329332 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
Apr 13 02:26:23.329332 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=106236) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 13 02:26:23.329332 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=106236) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 13 02:26:23.330301 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=106236) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 13 02:26:23.330301 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=106236) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 13 02:26:23.330301 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=106236) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 13 02:26:23.330301 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [-] Loading core plugin: ml2
Apr 13 02:26:23.612718 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.612863 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.612863 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.613554 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.613658 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.613658 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.648043 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.648173 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.648173 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.654096 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.654196 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.654196 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.829782 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.829902 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.829902 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.830356 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.830440 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.830586 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.905158 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7c15a1a6e340> agent after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.905350 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7c15a1a6e340> agent after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.905350 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x7c15a1a6e3e0> segment precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.905350 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x7c15a1a6e480> network precommit_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.905717 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x7c15a1a6f6a0> segment before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.958363 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> network before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.958581 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x7c15a1a62fc0> agent before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.958581 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> network before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.958581 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> port before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.958581 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> port before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.959224 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> segment precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.959224 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> segment precommit_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.959224 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> segment after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.959224 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> segment after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.959224 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> network after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960126 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> network precommit_delete 0, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960126 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> port provisioning_complete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960126 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960126 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960126 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> agent after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960719 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> subnet after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960719 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> subnet precommit_delete 0, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960719 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960719 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.960719 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> segment_host_mapping after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.961944 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> agent after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.961944 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.961944 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.961944 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c15a1ae0d70>> rbac-policy before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:23.961944 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:23.961944 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
Apr 13 02:26:23.962634 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
Apr 13 02:26:23.963654 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
Apr 13 02:26:23.963654 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
Apr 13 02:26:23.963654 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
Apr 13 02:26:23.963654 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
Apr 13 02:26:23.963654 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
Apr 13 02:26:24.000659 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
Apr 13 02:26:24.000659 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
Apr 13 02:26:24.001059 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
Apr 13 02:26:24.001384 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7c15a1422c30>> floatingip before_response 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.001548 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7c15a1422c30>> port before_response 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.030540 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
Apr 13 02:26:24.030697 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
Apr 13 02:26:24.030697 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
Apr 13 02:26:24.030841 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
Apr 13 02:26:24.080895 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO dbcounter [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Registered counter for database neutron
Apr 13 02:26:24.097239 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_db.sqlalchemy.engines [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106236) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
Apr 13 02:26:24.561894 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG dbcounter [-] [106236] Writer thread running {{(pid=106236) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Apr 13 02:26:24.754517 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Initializing driver for type 'local'
Apr 13 02:26:24.754749 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Initializing driver for type 'vlan'
Apr 13 02:26:24.865486 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] VlanTypeDriver initialization complete
Apr 13 02:26:24.865486 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Initializing extension driver 'port_security'
Apr 13 02:26:24.865486 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.extensions.port_security [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] PortSecurityExtensionDriver initialization complete
Apr 13 02:26:24.865782 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Initializing extension driver 'qos'
Apr 13 02:26:24.865782 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.plugins.ml2.extensions.qos [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QosExtensionDriver initialization complete {{(pid=106236) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
Apr 13 02:26:24.865782 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Initializing mechanism driver 'ovn'
Apr 13 02:26:24.865942 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Starting OVNMechanismDriver
Apr 13 02:26:24.867594 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> process before_spawn 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.867725 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> process after_init 55550000, True {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.867800 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> segment after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.867864 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> segment after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.867930 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> segment after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868037 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> allowed_address_pair before_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868125 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868231 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868321 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868409 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group precommit_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868494 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868578 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group_rule precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868660 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group_rule after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868752 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> security_group_rule before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868752 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> address_group after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868891 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> address_group after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.868975 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c15a17b64b0>> address_group after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.869111 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x7c15a17f4ad0>> qos_plugin after_init 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.869298 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x7c15a09dcf80>> trunk_plugin after_init 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.869476 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x7c15a09dcfb0>> logging-plugin after_init 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.870106 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.services.logapi.drivers.ovn.driver [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVN logging driver registered
Apr 13 02:26:24.871402 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.plugins.ml2.plugin [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=106236) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Apr 13 02:26:24.871630 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd280>> port after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.871725 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd280>> port after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.871789 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd280>> port after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.871853 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd190>> subnet after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872149 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd190>> subnet after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872149 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd190>> subnet after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872149 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd130>> network after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872400 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd130>> network after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872400 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd130>> network after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872400 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd100>> security_group after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872400 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd100>> security_group after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872630 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd100>> security_group after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872630 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd160>> security_group_rule after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872630 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd160>> security_group_rule after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872797 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd160>> security_group_rule after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872797 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd070>> address_group after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872918 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd070>> address_group after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.872987 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c15a09dd070>> address_group after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.873078 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ML2 OVO RPC backend initialized. {{(pid=106236) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
Apr 13 02:26:24.873821 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.quota [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7c15a09dd040>.
Apr 13 02:26:24.873898 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.plugin [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Modular L2 Plugin initialization complete
Apr 13 02:26:24.874031 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading services supported by the core plugin {{(pid=106236) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
Apr 13 02:26:24.874122 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.plugins.ml2.managers [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Got port-security extension from driver 'port_security'
Apr 13 02:26:24.874403 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=106236) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
Apr 13 02:26:24.874504 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: ovn-router
Apr 13 02:26:24.881890 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c15a09de930>> router before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882024 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x7c15a18cb380> port after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882098 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x7c15a18cb240> port after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882190 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x7c15a18cb2e0> subnet after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882268 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x7c15a18cb420> subnetpool_address_scope after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882351 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7c15a01af100> port before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882435 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7c15a01af100> port after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882512 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c15a09de930>> port precommit_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882600 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x7c15a18b32e0> port before_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882682 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x7c15a18b3240> port before_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882767 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c15a09de930>> router precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882852 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c15a09de930>> router precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.882930 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c15a09de930>> router precommit_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.883037 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x7c15a01af060> subnet after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.883616 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Tracking information for resource: router configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:24.883696 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.quota.resource_registry [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Tracking information for resource: floatingip configured {{(pid=106236) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 13 02:26:24.883762 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.services.ovn_l3.plugin [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Starting OVNL3RouterPlugin
Apr 13 02:26:24.883959 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c15a014bf80>> port_forwarding_plugin after_init 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.884148 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c15a09de780>> router before_create 55550100, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.884229 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c15a09de780>> router precommit_delete 55550100, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.884317 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c15a09de780>> router precommit_create 55550100, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.884402 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c15a09de780>> router precommit_update 55550100, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.886933 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.services.provider_configuration [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Service providers = [] {{(pid=106236) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
Apr 13 02:26:24.886988 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.db.servicetype_db [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=106236) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}}
Apr 13 02:26:24.888084 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router precommit_create 55549900, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888157 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router_interface after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888271 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> floatingip after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888367 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> floatingip precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888431 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> floatingip precommit_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888491 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> floatingip after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888583 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> floatingip after_status_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888647 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> floatingip after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888738 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> port_forwarding after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888815 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router_interface after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888921 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router after_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.888986 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.889085 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router after_delete 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.889148 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c15a0a0f620>> router after_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.889279 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.services.service_base [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=106236) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
Apr 13 02:26:24.889399 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c15a09de930>> process after_init 55550001, True {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.889463 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.889530 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: auto_allocate
Apr 13 02:26:24.890316 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7c15a01c9440> network precommit_update 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.890411 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron_lib.callbacks.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7c15a01c9440> network precommit_create 55550000, False {{(pid=106236) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:24.890572 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.890643 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: tag
Apr 13 02:26:24.892771 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.892853 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: timestamp
Apr 13 02:26:24.893939 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.894045 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: network_ip_availability
Apr 13 02:26:24.895283 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.895283 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: flavors
Apr 13 02:26:24.896123 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.896246 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading Plugin: revisions
Apr 13 02:26:24.896971 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.manager [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=106236) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 13 02:26:24.897109 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_concurrency.lockutils [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.568s {{(pid=106236) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 13 02:26:24.897307 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=106236) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
Apr 13 02:26:24.897407 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Initializing extension manager.
Apr 13 02:26:24.897668 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: __init__.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.897668 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: __pycache__ {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.897791 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: address_group.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.898081 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.898273 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: address-group
Apr 13 02:26:24.898360 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: address_scope.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.898601 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.898738 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: address-scope
Apr 13 02:26:24.898802 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: admin_state_down_before_update.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.899026 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.899177 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
Apr 13 02:26:24.899244 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: agent.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.899487 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.899631 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: agent
Apr 13 02:26:24.899696 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: agent_resources_synced.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.899894 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.900064 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension agent-resources-synced not supported by any of loaded plugins
Apr 13 02:26:24.900169 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: allowedaddresspairs.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.900674 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.900801 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: allowed-address-pairs
Apr 13 02:26:24.900862 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: auto_allocated_topology.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.901104 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.901244 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: auto-allocated-topology
Apr 13 02:26:24.901309 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: availability_zone.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.901541 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.901703 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: availability_zone
Apr 13 02:26:24.901784 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: availability_zone_filter.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.901980 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.902124 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension availability_zone_filter not supported by any of loaded plugins
Apr 13 02:26:24.902186 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: data_plane_status.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.902394 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.902522 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension data-plane-status not supported by any of loaded plugins
Apr 13 02:26:24.902585 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: default_subnetpools.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.902779 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.902915 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: default-subnetpools
Apr 13 02:26:24.902975 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: dhcpagentscheduler.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.903331 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.903459 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: dhcp_agent_scheduler
Apr 13 02:26:24.903546 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: dns.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.903720 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.903847 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: dns-integration
Apr 13 02:26:24.903912 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: dns_domain_ports.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.904144 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.904256 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: dns-domain-ports
Apr 13 02:26:24.904327 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.904533 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.904641 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: dns-integration-domain-keywords
Apr 13 02:26:24.904719 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: dvr.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.904942 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.905095 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension dvr not supported by any of loaded plugins
Apr 13 02:26:24.905160 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: ecmp_routes.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.905346 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.905480 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension ecmp_routes not supported by any of loaded plugins
Apr 13 02:26:24.905554 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: empty_string_filtering.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.905760 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.905895 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: empty-string-filtering
Apr 13 02:26:24.905972 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.906182 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.906316 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
Apr 13 02:26:24.906390 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.906574 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.906709 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
Apr 13 02:26:24.906770 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: external_net.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.906960 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.907114 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: external-net
Apr 13 02:26:24.907186 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: extra_dhcp_opt.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.907381 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.907505 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: extra_dhcp_opt
Apr 13 02:26:24.907576 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: extraroute.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.907758 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.907894 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: extraroute
Apr 13 02:26:24.907968 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: extraroute_atomic.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.908185 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.908318 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: extraroute-atomic
Apr 13 02:26:24.908400 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: filter_validation.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.908593 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.908725 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: filter-validation
Apr 13 02:26:24.908780 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: fip_pf_description.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.908991 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.909142 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
Apr 13 02:26:24.909215 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: fip_pf_detail.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.909403 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.909536 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: floating-ip-port-forwarding-detail
Apr 13 02:26:24.909638 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: fip_pf_port_range.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.909808 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.909943 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
Apr 13 02:26:24.910039 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: fip_port_details.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.910221 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.910353 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: fip-port-details
Apr 13 02:26:24.910424 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: flavors.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.910746 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.910887 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: flavors
Apr 13 02:26:24.910956 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.911243 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.911381 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
Apr 13 02:26:24.911456 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: floatingip_pools.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.911673 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.911810 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: floatingip-pools
Apr 13 02:26:24.911877 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: ip_allocation.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.912091 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.912217 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension ip_allocation not supported by any of loaded plugins
Apr 13 02:26:24.912288 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.912486 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.912612 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension ip-substring-filtering not supported by any of loaded plugins
Apr 13 02:26:24.912684 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l2_adjacency.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.912869 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.913021 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension l2_adjacency not supported by any of loaded plugins
Apr 13 02:26:24.913096 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.913436 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.913578 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: router
Apr 13 02:26:24.913668 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_conntrack_helper.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.914186 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.914328 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension l3-conntrack-helper not supported by any of loaded plugins
Apr 13 02:26:24.914399 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.914606 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.914744 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: enable-default-route-bfd
Apr 13 02:26:24.914796 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.915014 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.915159 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: enable-default-route-ecmp
Apr 13 02:26:24.915233 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.915425 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.915555 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: ext-gw-mode
Apr 13 02:26:24.915628 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.915813 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.915945 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: l3-ha
Apr 13 02:26:24.916034 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.916220 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.916351 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
Apr 13 02:26:24.916423 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_extra_gws.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.916605 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.916738 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: external-gateway-multihoming
Apr 13 02:26:24.916810 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_flavors.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.916990 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.917141 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: l3-flavors
Apr 13 02:26:24.917214 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_ndp_proxy.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.917459 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.917606 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension l3-ndp-proxy not supported by any of loaded plugins
Apr 13 02:26:24.917670 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.917856 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.917989 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
Apr 13 02:26:24.918079 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: l3agentscheduler.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.918523 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.918662 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension l3_agent_scheduler not supported by any of loaded plugins
Apr 13 02:26:24.918730 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: local_ip.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.919060 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.919323 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension local_ip not supported by any of loaded plugins
Apr 13 02:26:24.919323 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: logging.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.919466 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.919602 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension logging not supported by any of loaded plugins
Apr 13 02:26:24.919661 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: metering.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.920702 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.920830 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension metering not supported by any of loaded plugins
Apr 13 02:26:24.920898 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.921129 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.921267 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
Apr 13 02:26:24.921320 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: multiprovidernet.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.921525 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.921670 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: multi-provider
Apr 13 02:26:24.921743 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: netmtu.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.921938 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.922084 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: net-mtu
Apr 13 02:26:24.922156 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: netmtu_writable.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.922340 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.922473 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: net-mtu-writable
Apr 13 02:26:24.922545 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: network_availability_zone.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.922753 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.922889 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: network_availability_zone
Apr 13 02:26:24.922964 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: network_ha.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.923164 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.923298 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension network_ha not supported by any of loaded plugins
Apr 13 02:26:24.923375 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: network_ip_availability.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.923575 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.923706 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: network-ip-availability
Apr 13 02:26:24.923778 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: network_ip_availability_details.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.923977 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.924137 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: network-ip-availability-details
Apr 13 02:26:24.924194 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: network_segment_range.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.924466 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.924594 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension network-segment-range not supported by any of loaded plugins
Apr 13 02:26:24.924667 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: pagination.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.924861 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.924939 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: pagination
Apr 13 02:26:24.925034 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_device_profile.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.925224 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.925360 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: port-device-profile
Apr 13 02:26:24.925432 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_hardware_offload_type.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.925660 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.925864 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-hardware-offload-type not supported by any of loaded plugins
Apr 13 02:26:24.925864 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.926047 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.926209 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
Apr 13 02:26:24.926257 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_hints.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.926426 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.926592 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-hints not supported by any of loaded plugins
Apr 13 02:26:24.926639 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_mac_address_override.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.926809 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.926940 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-mac-override not supported by any of loaded plugins
Apr 13 02:26:24.927029 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.927212 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.927368 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: port-mac-address-regenerate
Apr 13 02:26:24.927411 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.927595 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.927728 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: port-numa-affinity-policy
Apr 13 02:26:24.927823 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.927985 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.928136 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: port-numa-affinity-policy-socket
Apr 13 02:26:24.928209 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_resource_request.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.928411 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.928534 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-resource-request not supported by any of loaded plugins
Apr 13 02:26:24.928593 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_resource_request_groups.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.928784 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.928920 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-resource-request-groups not supported by any of loaded plugins
Apr 13 02:26:24.928986 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: port_trusted_vif.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.929186 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.929320 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-trusted-vif not supported by any of loaded plugins
Apr 13 02:26:24.929410 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: portbindings.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.929574 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.929730 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: binding
Apr 13 02:26:24.929803 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: portbindings_extended.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.930020 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.930170 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: binding-extended
Apr 13 02:26:24.930244 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: portsecurity.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.930429 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.930563 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: port-security
Apr 13 02:26:24.930636 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: project_id.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.930825 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.930900 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: project-id
Apr 13 02:26:24.930979 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: providernet.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.931178 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.931315 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: provider
Apr 13 02:26:24.931388 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qinq.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.931589 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.931727 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: qinq
Apr 13 02:26:24.931795 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.932544 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.932686 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos not supported by any of loaded plugins
Apr 13 02:26:24.932751 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.932974 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.933130 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
Apr 13 02:26:24.933198 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.933393 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.933522 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
Apr 13 02:26:24.933619 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_default.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.933819 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.933952 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-default not supported by any of loaded plugins
Apr 13 02:26:24.934060 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_fip.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.934229 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.934364 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-fip not supported by any of loaded plugins
Apr 13 02:26:24.934436 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_fip_network_policy.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.934617 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.934751 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-fip-network-policy not supported by any of loaded plugins
Apr 13 02:26:24.934825 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_gateway_ip.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.935013 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.935159 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-gateway-ip not supported by any of loaded plugins
Apr 13 02:26:24.935231 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_port_network_policy.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.935416 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.935550 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-port-network-policy not supported by any of loaded plugins
Apr 13 02:26:24.935621 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.935813 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.935947 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-pps-minimum not supported by any of loaded plugins
Apr 13 02:26:24.936033 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.936260 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.936383 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
Apr 13 02:26:24.936437 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_pps_rule.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.936639 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.936772 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-pps not supported by any of loaded plugins
Apr 13 02:26:24.936843 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_rule_type_details.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.937049 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.937184 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-rule-type-details not supported by any of loaded plugins
Apr 13 02:26:24.937258 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_rule_type_filter.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.937443 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.937576 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-rule-type-filter not supported by any of loaded plugins
Apr 13 02:26:24.937656 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: qos_rules_alias.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.937877 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.938008 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension qos-rules-alias not supported by any of loaded plugins
Apr 13 02:26:24.938088 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: quota_check_limit.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.938273 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.938409 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: quota-check-limit
Apr 13 02:26:24.938482 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: quota_check_limit_default.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.938688 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.938820 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: quota-check-limit-default
Apr 13 02:26:24.938894 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: quotasv2.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.939309 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.939520 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: quotas
Apr 13 02:26:24.939600 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: quotasv2_detail.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.939897 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.939948 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: quota_details
Apr 13 02:26:24.940042 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: rbac.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.940420 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.940585 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: rbac-policies
Apr 13 02:26:24.940635 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: rbac_address_group.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.940839 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.940984 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension rbac-address-group not supported by any of loaded plugins
Apr 13 02:26:24.941070 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: rbac_address_scope.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.941267 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.941398 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: rbac-address-scope
Apr 13 02:26:24.941470 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: rbac_security_groups.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.941689 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.941814 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: rbac-security-groups
Apr 13 02:26:24.941885 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: rbac_subnetpool.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.942088 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.942234 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension rbac-subnetpool not supported by any of loaded plugins
Apr 13 02:26:24.942294 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: revisionifmatch.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.942478 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.942613 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: revision-if-match
Apr 13 02:26:24.942686 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: revisions.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.942882 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.943030 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: standard-attr-revisions
Apr 13 02:26:24.943106 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: router_availability_zone.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.943310 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.943446 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: router_availability_zone
Apr 13 02:26:24.943519 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: router_enable_snat.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.943702 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.943838 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: router-enable-snat
Apr 13 02:26:24.943911 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: routerservicetype.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.944113 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.944248 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension router-service-type not supported by any of loaded plugins
Apr 13 02:26:24.944348 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: security_groups_default_rules.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.944683 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.944822 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: security-groups-default-rules
Apr 13 02:26:24.944898 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.945104 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.945251 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: security-groups-normalized-cidr
Apr 13 02:26:24.945311 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: security_groups_port_filtering.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.945509 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.945663 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension port-security-groups-filtering not supported by any of loaded plugins
Apr 13 02:26:24.945738 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.945925 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.946083 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: security-groups-remote-address-group
Apr 13 02:26:24.946159 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.946338 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.946478 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: security-groups-rules-belongs-to-default-sg
Apr 13 02:26:24.946545 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.946723 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.946858 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: security-groups-shared-filtering
Apr 13 02:26:24.946929 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: securitygroup.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.948366 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.948511 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: security-group
Apr 13 02:26:24.948587 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: segment.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.948844 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.948984 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension segment not supported by any of loaded plugins
Apr 13 02:26:24.949067 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.949263 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.949395 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
Apr 13 02:26:24.949499 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: servicetype.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.949696 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.949822 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: service-type
Apr 13 02:26:24.949896 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: sorting.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.950119 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.950184 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: sorting
Apr 13 02:26:24.950254 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: standard_attr_segment.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.950437 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.950584 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension standard-attr-segment not supported by any of loaded plugins
Apr 13 02:26:24.950660 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: standardattrdescription.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.950844 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.950979 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: standard-attr-description
Apr 13 02:26:24.951067 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: stateful_security_group.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.951255 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.951392 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: stateful-security-group
Apr 13 02:26:24.951465 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: stdattrs_common.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.951639 np0000165409 neutron-ovn-maintenance-worker[106236]: WARNING neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
Apr 13 02:26:24.951697 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.951887 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.952043 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: subnet-dns-publish-fixed-ip
Apr 13 02:26:24.952118 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnet_external_network.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.952299 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.952436 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: subnet-external-network
Apr 13 02:26:24.952508 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnet_onboard.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.952704 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.952956 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension subnet_onboard not supported by any of loaded plugins
Apr 13 02:26:24.953029 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.953228 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.953366 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
Apr 13 02:26:24.953441 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnet_service_types.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.953639 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.953775 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: subnet-service-types
Apr 13 02:26:24.953847 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnetallocation.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.954069 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.954210 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: subnet_allocation
Apr 13 02:26:24.954354 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.954485 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.954622 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: subnetpool-prefix-ops
Apr 13 02:26:24.954693 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: tag_creation.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.954874 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.955012 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: tag-creation
Apr 13 02:26:24.955106 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.955288 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.955422 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
Apr 13 02:26:24.955496 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: tagging.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.956148 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.956287 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: standard-attr-tag
Apr 13 02:26:24.956335 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: timestamp.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.956563 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.956699 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: standard-attr-timestamp
Apr 13 02:26:24.956753 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: trunk.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.956962 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.957110 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension trunk not supported by any of loaded plugins
Apr 13 02:26:24.957183 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: trunk_details.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.957378 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.957505 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension trunk-details not supported by any of loaded plugins
Apr 13 02:26:24.957599 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: uplink_status_propagation.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.957793 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.957925 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension uplink-status-propagation not supported by any of loaded plugins
Apr 13 02:26:24.957997 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.958199 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.958335 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
Apr 13 02:26:24.958408 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loading extension file: vlantransparent.py {{(pid=106236) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 13 02:26:24.958610 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=106236) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 13 02:26:24.958740 np0000165409 neutron-ovn-maintenance-worker[106236]: INFO neutron.api.extensions [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Loaded extension: vlan-transparent
Apr 13 02:26:24.961199 np0000165409 neutron-ovn-maintenance-worker[106236]: WARNING oslo_service.backend._threading.service [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
Apr 13 02:26:24.961523 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Full set of CONF: {{(pid=106236) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
Apr 13 02:26:24.961573 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ******************************************************************************** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Apr 13 02:26:24.961650 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Configuration options gathered from: {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Apr 13 02:26:24.961741 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Apr 13 02:26:24.961843 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Apr 13 02:26:24.961887 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ================================================================================ {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Apr 13 02:26:24.962045 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] agent_down_time                = 75 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962137 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] allow_automatic_dhcp_failover  = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962227 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] allow_bulk                     = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962410 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962480 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] api_extensions_path            =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962573 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] api_paste_config               = api-paste.ini {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962668 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] api_workers                    = 4 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962758 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] auth_strategy                  = keystone {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962848 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] backlog                        = 4096 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.962940 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963074 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] bind_host                      = 0.0.0.0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963170 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] bind_port                      = 9696 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963259 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] client_socket_timeout          = 900 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963354 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] config_dir                     = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963422 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963512 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] config_source                  = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963605 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] control_exchange               = neutron {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963696 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] core_plugin                    = ml2 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963789 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] debug                          = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.963862 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] default_availability_zones     = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964040 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964165 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] dhcp_agent_notification        = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964235 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] dhcp_agents_per_network        = 1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964276 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] dhcp_lease_duration            = 86400 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964385 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] dhcp_load_type                 = networks {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964470 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] dns_domain                     = openstacklocal {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964564 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964650 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_default_route_bfd       = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964737 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_default_route_ecmp      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964824 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_dvr                     = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.964913 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_new_agents              = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965018 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965106 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_snat_by_default         = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965189 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] enable_traditional_dhcp        = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965273 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] external_dns_driver            = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965360 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] filter_validation              = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965451 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] global_physnet_mtu             = 1430 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965519 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] graceful_shutdown_timeout      = 60 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965716 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] host                           = np0000165409 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965818 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] host_dvr_for_dhcp              = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.965908 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] http_retries                   = 3 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966029 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] instance_format                = [instance: %(uuid)s]  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966115 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966204 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ipam_driver                    = internal {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966290 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ipv6_pd_enabled                = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966383 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] l3_ha                          = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966469 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966558 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] l3_ha_network_physical_name    =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966647 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] l3_ha_network_type             =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966738 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_color                      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966830 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_config_append              = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.966922 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967013 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_dir                        = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967118 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_file                       = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967184 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_options                    = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967274 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_rotate_interval            = 1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967369 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_rotate_interval_type       = days {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967458 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] log_rotation_type              = none {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967509 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967585 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967675 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967764 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967843 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967930 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_allowed_address_pair       = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.967994 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_dns_nameservers            = 5 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968107 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_header_line                = 16384 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968201 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_l3_agents_per_router       = 3 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968281 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_logfile_count              = 30 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968367 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_logfile_size_mb            = 200 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968455 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_routes                     = 30 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968540 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] max_subnet_host_routes         = 20 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968629 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] my_ip                          = 199.204.45.216 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968718 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:feb3:4c3f {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968805 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] network_auto_schedule          = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968888 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] network_link_prefix            = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.968985 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969099 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] notify_nova_on_port_data_changes = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969182 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] notify_nova_on_port_status_changes = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969271 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] pagination_max_limit           = -1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969357 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] periodic_fuzzy_delay           = 5 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969441 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] periodic_interval              = 40 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969524 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] publish_errors                 = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969626 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rate_limit_burst               = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969716 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rate_limit_except_level        = CRITICAL {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969802 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rate_limit_interval            = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969890 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] retry_until_window             = 30 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.969988 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] router_distributed             = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970074 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rpc_resources_processing_step  = 20 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970166 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rpc_response_max_timeout       = 600 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970250 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rpc_response_timeout           = 60 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970342 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rpc_state_report_workers       = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970426 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] rpc_workers                    = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970511 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] send_events_interval           = 2 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970605 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] service_plugins                = ['ovn-router'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970695 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] setproctitle                   = on {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970778 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] shell_completion               = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970865 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] state_path                     = /opt/stack/data/neutron {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.970955 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] syslog_log_facility            = LOG_USER {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971061 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] tcp_keepidle                   = 600 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971151 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] transport_url                  = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971237 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] use_journal                    = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971324 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] use_json                       = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971411 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] use_ssl                        = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971494 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] use_stderr                     = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971584 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] use_syslog                     = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971670 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] vlan_qinq                      = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971754 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] vlan_transparent               = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971840 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] wsgi_default_pool_size         = 100 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.971925 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] wsgi_keep_alive                = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.972035 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.972120 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] wsgi_server_debug              = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:24.972223 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_concurrency.disable_process_locking = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972317 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972422 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.connection_string     = messaging:// {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972516 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.enabled               = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972608 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.es_doc_type           = notification {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972699 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.es_scroll_size        = 10000 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972791 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.es_scroll_time        = 2m {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972883 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.filter_error_trace    = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.972976 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.hmac_keys             = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973084 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.sentinel_service_name = mymaster {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973176 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.socket_timeout        = 0.1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973272 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.trace_requests        = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973365 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler.trace_sqlalchemy      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973465 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler_jaeger.process_tags   = {} {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973554 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler_jaeger.service_name_prefix = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973646 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] profiler_otlp.service_name_prefix = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973754 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973843 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.973937 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974052 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974137 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974237 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.enforce_new_defaults = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974413 np0000165409 neutron-ovn-maintenance-worker[106236]: WARNING oslo_config.cfg [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
Apr 13 02:26:24.974508 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.enforce_scope      = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974603 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.policy_default_rule = default {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974704 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974796 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974890 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.974982 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975091 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975175 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975262 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975354 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_policy.remote_timeout     = 60.0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975454 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975550 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] service_providers.service_provider = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975651 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.auth_section              = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975750 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.auth_type                 = password {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975842 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.auth_url                  = https://199.204.45.216/identity {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.975930 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.cafile                    = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976035 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.certfile                  = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976137 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.collect_timing            = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976211 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.default_domain_id         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976298 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.default_domain_name       = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976383 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.domain_id                 = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976469 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.domain_name               = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976564 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.endpoint_type             = public {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976653 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.insecure                  = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976740 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.keyfile                   = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976831 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.password                  = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.976919 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.project_domain_id         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977013 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.project_domain_name       = Default {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977115 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.project_id                = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977208 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.project_name              = service {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977300 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.region_name               = RegionOne {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977389 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.split_loggers             = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977477 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.system_scope              = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977563 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.timeout                   = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977676 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.trust_id                  = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977763 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.user_domain_id            = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977853 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.user_domain_name          = Default {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.977946 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.user_id                   = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978057 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] nova.username                  = nova {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978152 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.auth_section         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978246 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.auth_type            = password {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978335 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.cafile               = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978417 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.certfile             = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978505 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.collect_timing       = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978599 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.endpoint_type        = public {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978689 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.insecure             = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978774 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.keyfile              = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978859 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.region_name          = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.978948 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.split_loggers        = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979052 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] placement.timeout              = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979151 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.auth_section            = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979235 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.auth_type               = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979323 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.cafile                  = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979405 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.certfile                = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979492 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.collect_timing          = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979578 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.connect_retries         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979664 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.connect_retry_delay     = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979751 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.enable_notifications    = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979838 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.endpoint_override       = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.979925 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.insecure                = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980029 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.interface               = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980116 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.keyfile                 = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980205 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.max_version             = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980297 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.min_version             = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980381 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.region_name             = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980467 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.retriable_status_codes  = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980552 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.service_name            = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980639 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.service_type            = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980725 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.split_loggers           = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980812 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.status_code_retries     = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980897 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.status_code_retry_delay = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.980982 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.timeout                 = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981085 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.valid_interfaces        = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981179 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ironic.version                 = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981266 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] cli_script.dry_run             = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981367 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981455 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981544 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981649 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981737 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981825 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.981933 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982060 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982092 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982194 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.hostname = np0000165409 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982281 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982368 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982459 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982550 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982655 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982727 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982815 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.982904 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983014 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983107 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983204 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983335 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983378 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983450 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983535 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983623 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983710 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983799 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983887 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.983974 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984081 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.ssl      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984174 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984273 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984357 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984444 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984535 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984623 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984717 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_notifications.driver = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984807 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_notifications.retry = -1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984906 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.984992 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_messaging_notifications.transport_url = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985102 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_reports.file_event_handler = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985193 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985288 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_reports.log_dir           = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985373 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985501 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.default_quota           = -1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985553 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985656 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_floatingip        = 50 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985744 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_network           = 100 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985832 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_port              = 500 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.985922 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986018 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_router            = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986115 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_security_group    = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986204 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986298 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.quota_subnet            = 100 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986380 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] QUOTAS.track_quota_usage       = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986478 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986567 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.group                  = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986656 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.helper_command         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986747 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.log_daemon_traceback   = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986838 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.986931 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.thread_pool_size       = 16 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987031 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep.user                   = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987128 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987214 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.group     = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987307 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.helper_command = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987392 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987482 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987569 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987654 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_dhcp_release.user      = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987747 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987832 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.group        = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.987918 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.helper_command = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988018 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988112 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988201 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988290 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_ovs_vsctl.user         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988401 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.capabilities = [21] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988458 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.group        = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988542 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.helper_command = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988629 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.log_daemon_traceback = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988717 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988804 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.thread_pool_size = 16 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988889 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_namespace.user         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.988981 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.capabilities = [12] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989084 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.group        = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989169 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.helper_command = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989257 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989349 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989433 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989516 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_conntrack.user         = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989656 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.capabilities      = [12, 21] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989742 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.group             = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989827 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.helper_command    = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.989913 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.log_daemon_traceback = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990015 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990117 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.thread_pool_size  = 16 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990202 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] privsep_link.user              = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990301 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990395 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990484 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] AGENT.use_helper_for_ns_read   = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990579 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] SECURITYGROUP.enable_ipset     = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990674 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] SECURITYGROUP.enable_security_group = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990762 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] SECURITYGROUP.firewall_driver  = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990853 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.990947 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] experimental.ipv6_pd_enabled   = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991068 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991158 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.external_network_type      = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991254 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991351 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.overlay_ip_version         = 4 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991439 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.path_mtu                   = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991529 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.physical_network_mtus      = {} {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991629 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.project_network_types      = ['geneve'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991720 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991818 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.991920 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992034 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2_type_geneve.max_header_size = 38 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992131 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992230 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992324 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] network_log.burst_limit        = 25 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992413 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] network_log.local_output_log_base = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992500 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] network_log.rate_limit         = 100 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992594 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.asyncio_connection    = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992680 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.asyncio_slave_connection = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992772 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.backend               = sqlalchemy {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992861 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.connection            = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.992951 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.connection_debug      = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993062 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.connection_parameters =  {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993153 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.connection_recycle_time = 3600 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993241 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.connection_trace      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993329 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.db_inc_retry_interval = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993418 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.db_max_retries        = 20 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993506 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.db_max_retry_interval = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993606 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.db_retry_interval     = 1 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993694 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.max_overflow          = 50 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993784 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.max_pool_size         = 5 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993871 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.max_retries           = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.993962 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994071 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.mysql_wsrep_sync_wait = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994154 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.pool_timeout          = None {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994263 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.retry_interval        = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994325 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.slave_connection      = **** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994412 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.sqlite_synchronous    = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994500 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.synchronous_reader    = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994587 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] database.use_db_reconnect      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994692 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.bfd_min_rx                 = 1000 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994779 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.bfd_min_tx                 = 100 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994868 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.bfd_mult                   = 3 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.994958 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995066 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995155 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995244 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.dns_records_ovn_owned      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995335 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.dns_servers                = [] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995432 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.enable_distributed_floating_ip = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995513 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.fdb_age_threshold          = 300 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995607 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ha_failover_strategy       = normal {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995702 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.live_migration_activation_strategy = rarp {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995805 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.localnet_learn_fdb         = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995897 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.mac_binding_age_threshold  = 300 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.995992 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.neutron_sync_mode          = log {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996101 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996193 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996284 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996377 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_metadata_enabled       = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996472 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996562 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996667 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_nb_connection          = ['ssl:199.204.45.216:6641'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996756 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996845 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_router_indirect_snat   = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.996937 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997045 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997142 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_sb_connection          = ['ssl:199.204.45.216:6642'] {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997234 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997322 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovs_create_tap             = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997421 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997509 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovsdb_log_level            = INFO {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997635 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997721 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997810 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.stateless_nat_enabled      = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.997901 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998014 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVS.bridge_mac_table_size      = 50000 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998109 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVS.igmp_flood                 = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998197 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVS.igmp_flood_reports         = True {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998287 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVS.igmp_flood_unregistered    = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998379 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVS.igmp_snooping_enable       = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998471 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] OVS.ovsdb_timeout              = 10 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998563 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998654 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998766 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:24.998809 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG cotyledon.oslo_config_glue [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] ******************************************************************************** {{(pid=106236) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Apr 13 02:26:24.998897 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG oslo_service.backend._threading.service [None req-1433d92d-be3e-406f-bca4-851c0f3c4278 None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x7c15a5fb6b10> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x7c15a5fb6b10> {{(pid=106236) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
Apr 13 02:26:25.031626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=107007) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
Apr 13 02:26:25.032037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Apr 13 02:26:25.032037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Apr 13 02:26:25.032037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Apr 13 02:26:25.032188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Apr 13 02:26:25.032188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Apr 13 02:26:25.032377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.032508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.032604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.032960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.033127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.033297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.033462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.033688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] backlog                        = 4096 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.033836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.034213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.034413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.034582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.034761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.034872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.035060 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.035235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.035387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.035546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.035723 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.036067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.036234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.036386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.036539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.036709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.036831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.037943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.038954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000165409 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039053 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.039923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.040948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.041912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.204.45.216 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:feb3:4c3f {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042827 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.042990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043281 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] retry_until_window             = 30 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.043956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044050 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044527 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] use_ssl                        = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.044906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.045092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.045092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.045204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.045294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 13 02:26:25.045449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.045589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.045733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.045959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.046924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047730 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.047972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.048965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.204.45.216/identity {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049259 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.049993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050318 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.050972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.051929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052723 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.052952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000165409 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.053955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054754 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.054811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.055953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.056945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.057947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058770 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.058953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.059996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.060963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061730 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061827 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.061919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.062938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063556 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.063957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.064927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065649 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.065989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.066968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.204.45.216:6641'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.204.45.216:6642'] {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.067960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 13 02:26:25.068923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=107007) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Apr 13 02:26:25.069191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [107007] {{(pid=107007) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
Apr 13 02:26:25.070984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-258313', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-553133'] for process (None), after_init {{(pid=107007) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 13 02:26:25.073205 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
Apr 13 02:26:25.189285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 13 02:26:25.189285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 13 02:26:25.189441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 13 02:26:25.189441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.189743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.190162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.190162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.190162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.192371 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.216:6641: connecting...
Apr 13 02:26:25.198768 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.216:6641: connected
Apr 13 02:26:25.234246 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
Apr 13 02:26:25.265286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 13 02:26:25.265370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 13 02:26:25.265433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.265515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=107007) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 13 02:26:25.267514 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.216:6642: connecting...
Apr 13 02:26:25.271953 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.216:6642: connected
Apr 13 02:26:25.309448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c15a0d7ad80>> floatingip precommit_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.309663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c15a0d7ad80>> floatingip precommit_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.309663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c15a0d7ad80>> floatingip before_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.309812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c15a0d7ad80>> floatingip before_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.309812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c15a0d7ad80>> port after_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.309926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c15a0d7ad80>> port precommit_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.310209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-58942'] for port_forwarding_plugin (None), after_init {{(pid=107007) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 13 02:26:25.310305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c15a014bf80>> port_forwarding after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.310386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c15a014bf80>> router_interface after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.310445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c15a014bf80>> port_forwarding after_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.310518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c15a014bf80>> port_forwarding after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.310577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c15a014bf80>> router_interface after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.310741 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 13 02:26:25.310809 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
Apr 13 02:26:25.311276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x7c15a0d6c350>> segment before_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.311424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x7c15a0d6c350>> subnet precommit_delete_associations 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.311691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> segment_host_mapping after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.311771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> segment_host_mapping after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.311831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> port after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.311898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> port after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.311955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> port after_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.312051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> segment after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.312110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> subnet after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.312180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> subnet after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.312244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c15a0e1c470>> subnet after_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.377315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c15a0da1df0>> subnet after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.377315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c15a0da1df0>> subnet after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.377588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c15a0da1df0>> subnet before_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.377588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c15a0da1df0>> subnet before_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.379917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7c15a01d8e60>> security_group after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.380115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7c15a01d8e60>> port after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.380185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1960987'] for logging-plugin (None), after_init {{(pid=107007) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 13 02:26:25.380330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c15a0ee1130>> security_group_rule after_create 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.380402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c15a0ee1130>> security_group_rule after_update 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:25.380487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c15a0ee1130>> security_group_rule after_delete 55550000, False {{(pid=107007) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 13 02:26:34.759433 np0000165409 neutron-ovn-maintenance-worker[106236]: DEBUG dbcounter [-] [106236] Writing DB stats neutron:SELECT=4 {{(pid=106236) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:26:35.381791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-04-13 02:26:35.381312 {{(pid=107007) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
Apr 13 02:26:35.384771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync port groups started @ 2026-04-13 02:26:35.384265 {{(pid=107007) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
Apr 13 02:26:35.403531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writer thread running {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Apr 13 02:26:35.415715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync port groups completed @ 2026-04-13 02:26:35.415354 {{(pid=107007) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
Apr 13 02:26:35.415936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-04-13 02:26:35.415715 {{(pid=107007) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1150}}
Apr 13 02:26:35.530430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.services.segments.db [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x7c15a0fa77d0>,) {'filters': {'network_id': ['034f578d-aed5-407a-803d-deb792507504'], 'is_dynamic': False}} {{(pid=107007) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Apr 13 02:26:35.535202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync metadata ports started {{(pid=107007) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1095}}
Apr 13 02:26:35.782473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync metadata ports completed {{(pid=107007) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1147}}
Apr 13 02:26:35.782473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=107007) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:947}}
Apr 13 02:26:35.837218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-04-13 02:26:35.836960 {{(pid=107007) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1020}}
Apr 13 02:26:35.837905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 13 02:26:35.838177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=107007) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1025}}
Apr 13 02:26:35.838256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=107007) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1084}}
Apr 13 02:26:35.838354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-04-13 02:26:35.838253 {{(pid=107007) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1319}}
Apr 13 02:26:35.838456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync ACLs started @ 2026-04-13 02:26:35.838367 {{(pid=107007) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
Apr 13 02:26:35.880418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync ACLs completed @ 2026-04-13 02:26:35.880178 {{(pid=107007) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:429}}
Apr 13 02:26:35.880588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync routers and router ports started @ 2026-04-13 02:26:35.880438 {{(pid=107007) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:734}}
Apr 13 02:26:35.931178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 13 02:26:35.931612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync routers and router ports completed @ 2026-04-13 02:26:35.931306 {{(pid=107007) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:942}}
Apr 13 02:26:35.931800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync port QoS policies started @ 2026-04-13 02:26:35.931610 {{(pid=107007) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1367}}
Apr 13 02:26:35.931957 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] Starting OVNClientQosExtension
Apr 13 02:26:35.948873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync port QoS policies completed @ 2026-04-13 02:26:35.948677 {{(pid=107007) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1390}}
Apr 13 02:26:35.948982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync Floating IP QoS policies started @ 2026-04-13 02:26:35.948851 {{(pid=107007) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1395}}
Apr 13 02:26:35.949192 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] Starting OVNClientQosExtension
Apr 13 02:26:35.969669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-04-13 02:26:35.969493 {{(pid=107007) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1418}}
Apr 13 02:26:35.969782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync Floating IP NAT rules started @ 2026-04-13 02:26:35.969629 {{(pid=107007) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1423}}
Apr 13 02:26:35.970070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-04-13 02:26:35.969859 {{(pid=107007) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1446}}
Apr 13 02:26:35.970070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d210eed7-3690-42a4-add9-84ad2456b75e None None] OVN-Northbound DB sync process completed @ 2026-04-13 02:26:35.969973 {{(pid=107007) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
Apr 13 02:26:45.384630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-04-13 02:26:45.383950 {{(pid=107007) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1474}}
Apr 13 02:26:45.385475 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 13 02:26:45.386972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] OVN-SB Sync hostname and physical networks started @ 2026-04-13 02:26:45.386654 {{(pid=107007) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1501}}
Apr 13 02:26:45.391619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.392828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.392936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.393983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.394178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.394721 np0000165409 neutron-ovn-maintenance-worker[107007]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] New host np0000165409 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
Apr 13 02:26:45.395051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.395247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.396935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.397096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.397209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.400876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=107007) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 13 02:26:45.402481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.404727 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
Apr 13 02:26:45.404830 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
Apr 13 02:26:45.412960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.413773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.413929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.415847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG oslo_concurrency.lockutils [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=107007) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 13 02:26:45.416164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG oslo_concurrency.lockutils [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=107007) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 13 02:26:45.420036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.421157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.425915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.430597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.431431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.431559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.440866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.441610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.450441 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
Apr 13 02:26:45.465166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.469984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.477940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.486557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron_lib.callbacks.manager [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-96025', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-1677067'] for segment_host_mapping (None), after_create {{(pid=107007) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 13 02:26:45.492228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:45.492699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.499494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:45.515339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Check for IGMP snoop support. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.516276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.516940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.530401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.538421 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_for_igmp_snoop_support finished in 0.001 seconds
Apr 13 02:26:45.539700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.540426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Check global DHCP options consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.541507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.546036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:45.547683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.551843 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
Apr 13 02:26:45.553163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.561252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.564669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.565292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:45.565499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.567144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.services.segments.db [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] Segments {'0d46232e-33db-40c1-91bf-c8d76fad6545'} mapped to the host np0000165409 {{(pid=107007) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
Apr 13 02:26:45.568102 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.585437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 13 02:26:45.587271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.588481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.611844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.620563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG oslo_concurrency.lockutils [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.172s {{(pid=107007) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 13 02:26:45.622220 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.028 seconds
Apr 13 02:26:45.625318 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.629888 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.176 seconds
Apr 13 02:26:45.649212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] OVN-SB Sync hostname and physical networks completed @ 2026-04-13 02:26:45.621687 {{(pid=107007) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1528}}
Apr 13 02:26:45.651452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:45.651764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.653219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:436}}
Apr 13 02:26:45.658547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.668853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.669100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.681137 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_for_inconsistencies finished in 0.114 seconds
Apr 13 02:26:45.685668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.686404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:45.691248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.692814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:45.699119 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.253 seconds
Apr 13 02:26:45.707697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.719725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.727990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.748728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: configure_nb_global {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.754542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:45.757402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.766160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.777830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.781385 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:45.784103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.799574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:45.804265 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.db.ovn_hash_ring_db [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] Cleaned up Hash Ring nodes older than 5 days
Apr 13 02:26:45.844459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.858029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.112 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:45.871807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:45.872345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:45.872724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.872724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:45.883038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.885711 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.095 seconds
Apr 13 02:26:45.887909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:45.892211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:26:45.894196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.897345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:45.914687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:45.915639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:45.916298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.916298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:45.916298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.940777 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.956764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.958117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:45.965331 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:45.968859 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:45.983387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:45.986843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:45.992916 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] OVN maintenance task set_network_type_and_physnet finished in 0.079 seconds
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:46.009768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.010385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:46.012648 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.012 seconds
Apr 13 02:26:46.021581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:46.023345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:46.047209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:46.056652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:46.061226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:46.073521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task configure_nb_global failed after 0.241 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 990, in configure_nb_global
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.080609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.081921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.082792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:46.082792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.082792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:46.085900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.091305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.100712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:46.105864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:46.108384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_router_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.119280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 990, in configure_nb_global
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:46.120147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.120834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.120834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:46.120834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.120834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:46.120834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:46.121548 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.517 seconds
Apr 13 02:26:46.124831 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.051 seconds
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.125551 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:46.126308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:46.129209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:46.132599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:46.136826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_router_static_routes {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:46.146986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.148311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:46.165770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.166643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:46.177151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=107007) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 13 02:26:46.179654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.184031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:46.184323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.314 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.194187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:46.194752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.195319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.195319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:46.195319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.195319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.197447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:46.198176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.198964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.198964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:46.198964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.198964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:46.199204 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_provider_distributed_ports finished in 0.588 seconds
Apr 13 02:26:46.200338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:46.202762 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_router_static_routes finished in 0.004 seconds
Apr 13 02:26:46.203979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.217094 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:46.227719 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task update_router_distributed_flag finished in 0.070 seconds
Apr 13 02:26:46.229747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.231062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.233034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.236648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:46.248489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.249492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:46.255298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.262 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.270108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.270929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.271858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:46.271858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.271858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:46.276881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Traceback (most recent call last):
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.279034 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.291025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:46.292237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.293425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.293425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:46.293425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.293425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:46.293425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] OVN maintenance task check_fdb_aging_settings failed after 0.814 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.304193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.304964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:46.305977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.305977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:46.306268 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] OVN maintenance task update_security_group_with_address_group finished in 0.084 seconds
Apr 13 02:26:46.308871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:46.315598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.316298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.317065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:46.317065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.317065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:46.317065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.318415 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:46.318993 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.654 seconds
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.218 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.322099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:46.322660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.323212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:46.325299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:46.326153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:46.326880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:46.326880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:46.326880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=107007) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 13 02:26:46.626058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 13 02:26:46.626731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-c4e742fa-e0c7-4b49-8f68-73c618bdc9ac None None] OVN-Southbound DB sync process completed @ 2026-04-13 02:26:46.626157 {{(pid=107007) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1482}}
Apr 13 02:26:47.029600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=a725b273-9f30-4992-9209-30b1f5a14a7c, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-034f578d-aed5-407a-803d-deb792507504', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-8dab20ec-1218-46be-8777-7aef7fbfc181', 'neutron:subnet_ids': 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:49:46:f0', name='lrp-1e3357f2-6d7e-43e6-92c2-a7eafd85c70a', networks=['fd89:d8a:5081::1/64'], options={}, peer=[], status={}) old= {{(pid=107007) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 13 02:26:47.030354 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 13 02:26:47.075226 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-034f578d-aed5-407a-803d-deb792507504 synchronized; highest priority chassis 75e1c21d-535b-42ed-8236-406877c7afe8
Apr 13 02:26:47.075702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '034f578d-aed5-407a-803d-deb792507504', 'neutron:router_id': '8dab20ec-1218-46be-8777-7aef7fbfc181'}),), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:47.075966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-034f578d-aed5-407a-803d-deb792507504, chassis=75e1c21d-535b-42ed-8236-406877c7afe8, priority=1, columns={}) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:47.076256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:47.076922 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2410, in link_network_ha_chassis_group
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 13 02:26:47.078229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise result.ex
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:47.078824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event 
Apr 13 02:26:49.022864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=6686d2dc-ff52-44d0-aa2f-c7b5a0a24482, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-8dab20ec-1218-46be-8777-7aef7fbfc181', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, gateway_chassis=[e3ea57b7-50bc-47c9-aed5-8d7325045ee8], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:ca:ba:c0', name='lrp-2646957a-8843-420b-bd80-c652210fce13', networks=['172.24.5.86/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '75e1c21d-535b-42ed-8236-406877c7afe8'}) old= {{(pid=107007) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 13 02:26:49.023950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=7602aa78-1b60-4776-a719-c6e8ac81a2f0, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-8dab20ec-1218-46be-8777-7aef7fbfc181', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a'}, gateway_chassis=[e3ea57b7-50bc-47c9-aed5-8d7325045ee8], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:ca:ba:c0', name='lrp-2646957a-8843-420b-bd80-c652210fce13', networks=['172.24.5.86/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=107007) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 13 02:26:49.059628 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-034f578d-aed5-407a-803d-deb792507504 synchronized; highest priority chassis 75e1c21d-535b-42ed-8236-406877c7afe8
Apr 13 02:26:49.060553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '034f578d-aed5-407a-803d-deb792507504', 'neutron:router_id': '8dab20ec-1218-46be-8777-7aef7fbfc181'}),), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:49.060938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-034f578d-aed5-407a-803d-deb792507504, chassis=75e1c21d-535b-42ed-8236-406877c7afe8, priority=1, columns={}) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:49.061255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.062077 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2410, in link_network_ha_chassis_group
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:49.063183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise result.ex
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.064078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event 
Apr 13 02:26:49.090509 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-034f578d-aed5-407a-803d-deb792507504 synchronized; highest priority chassis 75e1c21d-535b-42ed-8236-406877c7afe8
Apr 13 02:26:49.090916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '034f578d-aed5-407a-803d-deb792507504', 'neutron:router_id': '8dab20ec-1218-46be-8777-7aef7fbfc181'}),), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:49.091379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-034f578d-aed5-407a-803d-deb792507504, chassis=75e1c21d-535b-42ed-8236-406877c7afe8, priority=1, columns={}) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:49.091710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.092698 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2410, in link_network_ha_chassis_group
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:49.093660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise result.ex
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:49.094482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event 
Apr 13 02:26:50.430625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:50.431558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:50.431669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:50.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:50.463231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:50.463421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.463950 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.464966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.465553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:50.466175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.466175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:50.466459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.467079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.467643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:50.467643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.467643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:50.629845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:50.630942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:50.631156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:50.632276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:50.632701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.633900 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.635620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.635986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:50.636410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.636410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.637240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.637766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:50.638339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.638339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:50.750248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:50.751324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:50.751575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:50.751688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:26:50.752602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:50.753140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.754067 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:50.755562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.756348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.756901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:50.757482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.758109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.758109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:50.758109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.758109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.758494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:50.759349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.760322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.760322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:50.760322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.760322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:50.899204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:50.900083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:50.900145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:50.900266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:50.900742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:50.901064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.901718 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:50.902805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.903255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.904767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.905159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.905546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:50.905546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.905546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.906823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:50.907291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.907824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.907824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:50.907824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.907824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:50.983204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:50.984032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:50.984142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:50.984355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:50.984800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:50.985240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:50.985429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.986167 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.987154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:50.987863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:50.988523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:50.989359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:50.990249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:50.990249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:50.990249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:55.432221 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:55.433246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:55.435151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:55.436055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:55.468426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:55.468826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.470990 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.472157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.472450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:55.472988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.472988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:55.473334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.473705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.474340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:55.474340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.474340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:55.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:55.632629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:55.632710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:55.635042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:55.635301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.637058 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.637953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.638263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:55.638695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.638695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.639451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.639824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:55.640278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.640278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:55.751563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:55.752828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:55.752942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:55.753027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:26:55.753765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:55.754165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.754632 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:55.755609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.756365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.756716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:55.757282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.757901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:55.758625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.759921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:55.900442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:55.901440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:55.901570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:55.901806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:55.902627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:55.903360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.904256 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:26:55.905562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.906344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.906893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.907544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.908211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:55.908211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.908211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.909558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:55.910310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.911258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.911258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:55.911258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.911258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:26:55.984426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:26:55.985165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:26:55.985266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:26:55.985507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:26:55.986214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:55.986683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:26:55.986880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.987745 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.988794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:26:55.989394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:26:55.989980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:26:55.990702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:26:55.991545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:26:55.991545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:26:55.991545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:00.439155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:00.443158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:00.443158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:00.446404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:00.484710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:00.485219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.486355 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.487190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.487759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:00.488382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:00.489139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.490077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:00.490077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.490077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.490077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:00.490077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.490077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:00.632996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:00.634325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:00.634473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:00.635985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:00.636637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.637681 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.640393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.641022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:00.641630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.641630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.642360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.643176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:00.644066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.644066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:00.753190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:00.754417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:00.754604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:00.754751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:00.755551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:00.755913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.756798 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:00.758298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.759180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.760074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:00.760799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.761404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:00.762719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.764032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:00.901751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:00.902472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:00.902549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:00.902730 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:00.903267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:00.903611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.904287 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:00.905053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.905381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.905905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.906191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.906477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:00.906477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.906477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.907023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:00.907379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.907763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.907763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:00.907763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.907763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:00.985661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:00.986685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:00.986825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:00.987094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:00.987626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:00.987988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:00.988235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.989040 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.990496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:00.990779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.991053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:00.992192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:00.992541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:00.992896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:00.992896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:05.442722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:05.444569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:05.444752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:05.446296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:05.480810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:05.481282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.482208 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.483267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.483657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:05.484149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.484149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:05.484803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.485294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.485825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:05.485825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.485825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:05.634636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:05.636090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:05.636231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.639461 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.640432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.641081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.641765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:05.642610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.643634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.643634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:05.643634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.643634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:05.755319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:05.756643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:05.756787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:05.757031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:05.758017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:05.758463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.759376 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:05.760505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.761350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:05.762106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.762801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:05.763439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.764344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:05.902939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:05.904282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:05.904480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:05.904727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:05.905786 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:05.906455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.907360 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:05.908944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.910228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.910926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.911508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.912149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:05.912149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.912149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.912463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:05.913061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.913612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.913612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:05.913612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.913612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:05.986973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:05.988554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:05.988662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:05.988991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:05.989831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:05.990295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:05.990550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.991608 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.993590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:05.994280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.994873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:05.995920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:05.996703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:05.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:05.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:10.444615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:10.446108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:10.446302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:10.447573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:10.481330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.482544 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.483360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.483969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:10.484617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:10.485371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.486281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:10.486281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.486281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.486281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:10.486281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.486281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:10.636139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:10.637136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:10.637263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:10.638429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:10.638813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.639764 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.642355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.643052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.643653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:10.644434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.645370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.645370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:10.645370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.645370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:10.757202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:10.758734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:10.758938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:10.759062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:10.759787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:10.760185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.761088 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:10.762449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.763149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.763579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:10.764073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.764521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:10.764974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.765548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:10.904486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:10.905923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:10.906153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:10.906357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:10.907176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:10.907701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.908584 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:10.910226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.910739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.911216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.911764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.912189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:10.912189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.912189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.912517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:10.913096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.913677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.913677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:10.913677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.913677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:10.988673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:10.990037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:10.990205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:10.990482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:10.992121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:10.992443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:10.992443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.993269 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.994773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:10.995526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.996147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:10.996465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:10.997229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:10.998090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:10.998090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:15.446253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:15.447749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:15.447952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:15.449236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:15.479082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:15.479355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.479954 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.480708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.480982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:15.481280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.481280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:15.481498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.481838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.482681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:15.482681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.482681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:15.637669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:15.638830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:15.638946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:15.640152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:15.640414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.641117 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.642980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.643669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:15.644326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.644326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.644583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.645494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:15.646362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.646362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:15.758995 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:15.760693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:15.761042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:15.761042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:15.761862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:15.762136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.762978 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:15.764527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.765578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.766156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:15.766818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.767550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.767550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:15.767550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.767550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.768024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:15.768827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.769466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.769466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:15.769466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.769466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:15.906149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:15.907581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:15.907761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:15.907973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:15.909168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:15.909601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.910569 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:15.912237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.913068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.913906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.914526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.915177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:15.915177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.915177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.916023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:15.916836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.917755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.917755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:15.917755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.917755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:15.991412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:15.992366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:15.992568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:15.992740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:15.993448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:15.993895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:15.994131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.995372 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.996752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:15.997513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.998182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:15.998331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:15.999063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:15.999541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:15.999541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:20.449353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:20.450362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:20.450362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:20.451226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:20.482433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:20.482857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.483754 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.484578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.485317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:20.486044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:20.486825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.487829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:20.487829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.487829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.487829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:20.487829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.487829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:20.639427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:20.641641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:20.641794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:20.642858 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:20.643274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.644259 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.645855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.650236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.651482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:20.652391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.653471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.653471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:20.653471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.653471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:20.761038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:20.762530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:20.762683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:20.762839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:20.763666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:20.764104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.765233 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:20.766749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.767291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.768199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:20.768571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.768997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.768997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:20.768997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.768997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.769698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:20.770240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.770817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.770817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:20.770817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.770817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:20.908278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:20.908990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:20.909192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:20.909418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:20.910174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:20.910522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.911348 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:20.912853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.913384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.914397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.914788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.915211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:20.915211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.915211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:20.916209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:20.916724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:20.917353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:20.917353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:20.917353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:20.917353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:20.993147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:20.997807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:20.998176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:20.998608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:20.999582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:20.999922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:21.000108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:21.000853 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:21.002405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:21.002802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:21.003092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:21.004813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:21.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:21.005787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:21.005787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:25.449387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:25.450420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:25.450574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:25.452076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:25.483639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:25.484126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.485287 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.486333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.486797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:25.487259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:25.487779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.488441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:25.488441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.488441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.488441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:25.488441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.488441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:25.641969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:25.643988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:25.643988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:25.645447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:25.645447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.645794 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.646731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.647432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.648097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:25.648960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.650186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.650186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:25.650186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.650186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:25.762721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:25.763918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:25.764139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:25.764294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:25.766487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:25.766487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.766940 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:25.768254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.768803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.769497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:25.769914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.770377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.770377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:25.770377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.770377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.770722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:25.771292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.771888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.771888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:25.771888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.771888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:25.909127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:25.911089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:25.911089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:25.911089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:25.911612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:25.912209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.913022 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:25.913929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.914423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.914836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.915240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.915636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:25.915636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.915636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:25.915955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:25.916501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:25.917019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:25.917019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:25.917019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:25.917019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:25.997941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:25.998751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:25.998873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:25.999136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:25.999783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:26.000227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:26.000505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:26.001572 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:26.003121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:26.003719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:26.004350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:26.004512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:26.005284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:26.005959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:26.005959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:30.450555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:30.451871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:30.452264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:30.454200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:30.488773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:30.489223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.490054 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.491068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.491651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:30.492282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:30.492973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.493858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:30.493858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.493858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.493858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:30.493858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.493858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:30.643150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:30.644324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:30.644468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:30.645358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:30.645806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.646693 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.648311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.648867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:30.649430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.649430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.649918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.650623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:30.651219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.651219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:30.763954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:30.765223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:30.765412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:30.765537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:30.766216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:30.766542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.767367 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:30.768853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.769833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.770351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:30.771123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.771910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:30.773152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.774203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:30.910798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:30.912041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:30.912268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:30.912510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:30.913546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:30.914024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.914925 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:30.916753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.917411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.918600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.919145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.919805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:30.919805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.919805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:30.921097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:30.921874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:30.922827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:30.922827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:30.922827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:30.922827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:30.999183 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:31.000129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:31.000223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:31.000414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:31.000946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:31.001378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:31.001587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:31.002366 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:31.003514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:31.004125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:31.004677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:31.005391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:31.006163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:31.006163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:31.006163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:35.452036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:35.453234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:35.453430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:35.454643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:35.456522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=577,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:27:35.498574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:35.498791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.499463 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.500542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.501238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:35.501826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:35.502607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.503546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:35.503546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.503546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.503546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:35.503546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.503546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:35.644461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:35.645560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:35.645778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:35.646964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:35.647378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.648555 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.650202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:35.651394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.651394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.651993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.652863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:35.653809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.653809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:35.765178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:35.766248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:35.766426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:35.766544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:35.767146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:35.767514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.768298 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:35.769656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.770348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.771663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:35.772221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.772828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.772828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:35.772828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.772828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.773285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:35.774062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.774823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.774823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:35.774823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.774823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:35.912160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:35.913409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:35.913531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:35.913700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:35.914474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:35.914893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.915753 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:35.917041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.917897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.919224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:35.920042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:35.920792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:35.921697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:36.000153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:36.001034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:36.001135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:36.001362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:36.001840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:36.002212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:36.002407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:36.003201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:36.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:36.005201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:36.005847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:36.006032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:36.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:36.008092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:36.008092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:40.453553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:40.454865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:40.455185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:40.456503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:40.493427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:40.494025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.495176 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.496069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.496733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:40.497576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:40.498514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.499775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:40.499775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.499775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.499775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:40.499775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.499775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:40.645657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:40.646964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:40.647153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:40.648320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:40.648579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.649220 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.650869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.651433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:40.651932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.651932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.652597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.653406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:40.654437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.654437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:40.766475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:40.767903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:40.768040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:40.768231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:40.769094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:40.769486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.770471 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:40.771760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.772621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.772975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:40.773532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.774183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.774183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:40.774183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.774183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.774598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:40.775646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.776368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.776368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:40.776368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.776368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:40.914354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:40.915518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:40.915660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:40.915830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:40.916610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:40.917027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.917880 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:40.919332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.920066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.920692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.921251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.921852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:40.921852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.921852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:40.922405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:40.922945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:40.923460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:40.923460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:40.923460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:40.923460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:41.001214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:41.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:41.002355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:41.002621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:41.003251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:41.003536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:41.003772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:41.004345 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:41.005830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:41.006500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:41.007138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:41.007508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:41.008047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:41.008525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:41.008525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:45.455044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:45.457293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:45.457520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:45.458950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:45.493261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:45.493731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.494693 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.496292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.496915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:45.497488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:45.498157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.499068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:45.499068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.499068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.499068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:45.499068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.499068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:45.647337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:45.648191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:45.648323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:45.649241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:45.649583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.650469 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.652024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.652363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:45.652755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.652755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.653639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.654284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:45.654807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.654807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:45.768348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:45.769270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:45.769419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:45.769558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:45.770290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:45.770701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.771627 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:45.772926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.773491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.774216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:45.774523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.774907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.774907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:45.774907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.774907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.775820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:45.776358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.776825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.776825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:45.776825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.776825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:45.915788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:45.916696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:45.916837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:45.917023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:45.917735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:45.918178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.919053 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:45.920403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.921131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.922149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.922481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:45.922481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.922481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:45.923527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:45.924139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:45.924755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:45.924755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:45.924755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:45.924755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:46.002531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:46.003209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:46.003367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:46.003630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:46.004354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:46.004709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:46.004896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:46.005719 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:46.007398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:46.008054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:46.008631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:46.009165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:46.010960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:46.011936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:46.011936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:50.457255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:50.458092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:50.458176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:50.459476 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:50.489393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.507039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.507771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:50.508428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.509047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.509906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:50.648434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.689039 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.690697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.691254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.691795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:50.692505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.693373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.693373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:50.693373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.693373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:50.769407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:50.770251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:50.770436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:50.770609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:50.771580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:50.771987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.772993 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:50.774543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.775424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.775776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:50.776323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.777107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:50.777766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.778615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:50.916840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:50.917489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:50.917635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:50.917807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:50.918883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:50.919405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.920461 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:50.922048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.922789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.923819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.924463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.925131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:50.925131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.925131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:50.926138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:50.927100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:50.928021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:50.928021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:50.928021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:50.928021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:51.003132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:51.003669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:51.003809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:51.004101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:51.004975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:51.005524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:51.005945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:51.006998 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:51.008676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:51.009415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:51.010057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:51.010129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:51.011171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:51.012206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:51.012206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:55.458364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:55.459131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:55.459254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:55.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:55.486970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:55.487542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.488704 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.489564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.489955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:55.490318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.490318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:55.490382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.490811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.491350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:55.491350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.491350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:55.649508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:55.650735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:55.650887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:55.652148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:55.652490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.653235 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.655654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.656329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:55.656769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.656769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.657582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.658227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:55.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:55.770459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:55.771208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:55.771308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:55.771485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:27:55.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:55.772499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.773470 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:55.774895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.775851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.776387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:55.777133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.778171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:55.778689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.779384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:55.917601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:55.918275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:55.918388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:55.918450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:55.918880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:55.919208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.919698 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:27:55.920432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.921125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.921458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.922029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:55.922593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:55.923289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:55.924083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:27:56.003787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:27:56.004924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:27:56.004924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:27:56.004924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:27:56.005538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:56.006028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:27:56.006264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:56.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:56.008737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:27:56.009383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:56.009973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:27:56.010270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:27:56.011023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:27:56.011819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:27:56.011819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:00.459274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:00.460134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:00.460134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:00.461418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:00.498188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:00.498648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.499628 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.500729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.501311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:00.501890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:00.502419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.503363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:00.503363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.503363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.503363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:00.503363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.503363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:00.650712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:00.651479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:00.651650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:00.652879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:00.653281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.654234 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.656392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.656924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:00.657980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.657980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.658121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.658888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:00.660661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.660661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:00.771370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:00.772393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:00.772694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:00.772840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:00.773648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:00.773958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.774880 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:00.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.777575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.777913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:00.778396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.778933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.778933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:00.778933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.778933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.779317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:00.780064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.781080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.781080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:00.781080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.781080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:00.918432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:00.918781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:00.918898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:00.919119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:00.919799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:00.920234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.920985 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:00.922464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.923867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:00.925039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:00.925943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.926516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:00.928113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:01.004496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:01.005215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:01.005379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:01.005700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:01.006328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:01.006691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:01.006888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:01.007695 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:01.009351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:01.009970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:01.010958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:01.012536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:01.014476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:01.014476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:01.014476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:05.459985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:05.461805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:05.462085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:05.463324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:05.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:05.503021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.504229 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.505696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.506086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:05.507164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:05.507866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.508508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:05.508508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.508508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.508508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:05.508508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.508508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:05.651645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:05.652437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:05.652569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:05.653936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:05.654361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.655285 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.656958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.657855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:05.658400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.658400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.658612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.659779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:05.661307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.661307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:05.772096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:05.772750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:05.772902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:05.773046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:05.773783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:05.774215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.775478 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:05.776575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.777360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.777729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:05.778467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.779110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:05.779825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.780710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:05.918989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:05.919462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:05.919652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:05.920170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:05.921311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:05.921739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.922706 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:05.924158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.924710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.925672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.926096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.926486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:05.926486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.926486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:05.927422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:05.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:05.928644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:05.928644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:05.928644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:05.928644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:06.005024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:06.005637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:06.005778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:06.006017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:06.006984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:06.007465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:06.007698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:06.008683 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:06.010215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:06.010935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:06.011713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:06.012593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:06.013561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:06.013561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:06.013561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:10.461751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:10.462450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:10.462450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:10.463831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:10.500744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:10.501257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.502442 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.503890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.504537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:10.505272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:10.505766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.506252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:10.506252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.506252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.506252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:10.506252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.506252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:10.652360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:10.652903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:10.653057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:10.654512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:10.655072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.656520 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.657887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.658439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.658990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:10.659818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.660788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.660788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:10.660788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.660788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:10.772875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:10.773465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:10.773611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:10.773883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:10.774904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:10.775354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.776370 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:10.777940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.778769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.779277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:10.779738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.780154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.780154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:10.780154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.780154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.780943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:10.781388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.781892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.781892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:10.781892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.781892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:10.919881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:10.920862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:10.920991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:10.921229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:10.922440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:10.922902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.923941 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:10.925396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.926229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.926754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.927341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.927794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:10.927794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.927794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:10.928743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:10.929347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:10.929875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:10.929875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:10.929875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:10.929875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:11.005692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:11.005935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:11.006024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:11.006164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:11.007029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:11.007555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:11.007772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:11.009200 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:11.010764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:11.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:11.011896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:11.012334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:11.013100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:11.013894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:11.013894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:15.462539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:15.463290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:15.463290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:15.464681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:15.523374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:15.523739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.524868 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.525815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.526478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:15.527361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:15.528212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.529553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:15.529553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.529553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.529553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:15.529553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.529553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:15.653027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:15.653409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:15.653530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:15.654427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:15.654730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.657170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.658040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.658632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:15.659345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.660293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.660293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:15.660293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.660293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:15.773692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:15.774532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:15.774682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:15.774848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:15.775680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:15.776029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.776891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:15.778431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.779157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.779812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:15.780395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.780978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.780978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:15.780978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.780978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.781486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:15.782080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.782589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.782589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:15.782589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.782589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:15.921059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:15.921949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:15.922145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:15.922338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:15.922940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:15.923458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.924348 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:15.925281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.926045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.926390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.926936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:15.927552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:15.928305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:15.929067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:16.006416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:16.007056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:16.007214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:16.007465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:16.008026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:16.008452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:16.008674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:16.009405 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:16.010962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:16.011674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:16.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:16.012721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:16.013527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:16.014392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:16.014392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:20.463118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:20.463829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:20.463829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:20.465110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:20.517273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:20.517631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.518599 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.519720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.520655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:20.521319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:20.521959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.522809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:20.522809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.522809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.522809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:20.522809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.522809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:20.653681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:20.654305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:20.654357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:20.655328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:20.655767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.656684 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.658529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.658866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:20.659219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.659219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.660436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.660895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:20.661668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.661668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:20.774572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:20.775146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:20.775295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:20.775490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:20.776254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:20.776633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.777433 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:20.778862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.779550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.780547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:20.780999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.781399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.781399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:20.781399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.781399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.782385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:20.782826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.783342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.783342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:20.783342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.783342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:20.922132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:20.922549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:20.922684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:20.922850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:20.923650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:20.924191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.925045 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:20.926549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.927381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.928161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.928706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.929182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:20.929182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.929182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:20.929531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:20.930017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:20.930501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:20.930501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:20.930501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:20.930501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:21.007078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:21.007499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:21.007641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:21.007997 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:21.008600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:21.009120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:21.009385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:21.010316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:21.011834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:21.012325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:21.012697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:21.013394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:21.013830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:21.014304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:21.014304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:25.463754 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:25.464488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:25.464720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:25.466213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:25.503032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:25.503543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.504592 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.505550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.506154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:25.506727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:25.507478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.508392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:25.508392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.508392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.508392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:25.508392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.508392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:25.654603 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:25.655424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:25.655574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:25.657151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:25.657639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.659383 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.660933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.661314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:25.661636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.661636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.662560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.663022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:25.663491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.663491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:25.775715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:25.776481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:25.776673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:25.776853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:25.777526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:25.777760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.778448 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:25.780044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.780525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.781498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:25.782091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.782696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.782696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:25.782696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.782696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.783365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:25.784162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.784989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.784989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:25.784989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.784989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:25.923286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:25.924193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:25.924356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:25.924578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:25.925166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:25.925426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.926099 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:25.927660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.928432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.929523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.930163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.930823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:25.930823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.930823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:25.931303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:25.932194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:25.933239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:25.933239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:25.933239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:25.933239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:26.007761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:26.008049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:26.008125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:26.008285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:26.008607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:26.008848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:26.008957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:26.009442 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:26.010204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:26.011368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:26.011901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:26.012641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:26.013423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:26.013423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:26.013423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:30.464449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:30.465154 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:30.465433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:30.467007 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:30.499628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:30.500188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.501061 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.501822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.502515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:30.503304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:30.504279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.505034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:30.505034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.505034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.505034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:30.505034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.505034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:30.655485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:30.656128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:30.656278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:30.657820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:30.657820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.658371 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.659776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.660184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:30.660501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.660501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.661547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.662289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:30.662787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.662787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:30.776527 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:30.777302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:30.777466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:30.777601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:30.778272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:30.779472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.779685 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:30.781176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.781980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.782687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:30.783854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.785139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:30.785436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.785758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:30.924211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:30.924736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:30.924878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:30.925128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:30.927067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:30.927067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.927489 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:30.928693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.929377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.930193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.930701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.931312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:30.931312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.931312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:30.932100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:30.932561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:30.933044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:30.933044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:30.933044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:30.933044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:31.008317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:31.008817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:31.008924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:31.009202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:31.011160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:31.011160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:31.011160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:31.011346 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:31.012692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:31.013366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:31.013684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:31.014296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:31.015255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:31.015730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:31.015730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:35.465202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:35.465890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:35.466151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:35.467367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:35.492254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=254 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:28:35.501589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:35.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.503297 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.504292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.504818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:35.505380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:35.505962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.506624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:35.506624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.506624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.506624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:35.506624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.506624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:35.656332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:35.656815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:35.656952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:35.657899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:35.658252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.658993 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.660520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.661258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.661872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:35.662728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.663775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.663775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:35.663775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.663775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:35.777317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:35.777925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:35.778101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:35.778277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:35.778923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:35.779266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.780121 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:35.781540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.782283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.782825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:35.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.784109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.784109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:35.784109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.784109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.784488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:35.785273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:35.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:35.925094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:35.925501 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:35.925649 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:35.925822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:35.926693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:35.927131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.927794 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:35.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.929989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.930652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.931277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.931973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:35.931973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.931973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:35.932491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:35.933361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:35.934274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:35.934274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:35.934274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:35.934274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:36.008870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:36.009442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:36.009609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:36.009884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:36.010521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:36.010930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:36.011150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:36.011962 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:36.013537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:36.014162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:36.014747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:36.015682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:36.016864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:36.017946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:36.017946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:40.466132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:40.467140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:40.467140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:40.468151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:40.500262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:40.500690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.501615 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.502557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.503145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:40.503751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:40.504454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.505334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:40.505334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.505334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.505334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:40.505334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.505334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:40.657162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:40.657818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:40.657968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:40.659058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:40.659341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.660206 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.661833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.662496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.663172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:40.663970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.664950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.664950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:40.664950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.664950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:40.778068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:40.778550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:40.778691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:40.778847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:40.779504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:40.779899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.780767 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:40.782088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.782576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.783156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:40.783502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.783904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.783904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:40.783904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.783904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.784163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:40.784655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.785234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.785234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:40.785234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.785234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:40.925848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:40.926310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:40.926458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:40.926626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:40.927618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:40.927976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.928818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:40.930251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.930760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.931676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.932054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.932411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:40.932411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.932411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:40.933379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:40.933860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:40.934369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:40.934369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:40.934369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:40.934369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:41.009626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:41.010260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:41.010413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:41.010671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:41.011151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:41.011407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:41.011535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:41.012198 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:41.013744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:41.014145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:41.014500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:41.015326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:41.015821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:41.016345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:41.016345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:45.466732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:45.467204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:45.467394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:45.468312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:45.498830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:45.498939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.499777 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.501596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.502243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:45.502792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.502792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:45.503428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.504220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.505044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:45.505044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.505044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:45.657890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:45.658542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:45.658718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:45.659847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:45.660216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.661121 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.662774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.663338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:45.663882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.663882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.664532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.665293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:45.666098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.666098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:45.779090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:45.779584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:45.779730 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:45.779876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:45.780601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:45.780969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.781915 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:45.783278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.783964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.784563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:45.785178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.785798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.785798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:45.785798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.785798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.786716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:45.787462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.788416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.788416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:45.788416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.788416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:45.926894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:45.927650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:45.927813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:45.928069 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:45.928779 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:45.929259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.930129 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:45.931547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.932251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.933285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.933866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.934447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:45.934447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.934447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:45.935625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:45.936356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:45.937295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:45.937295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:45.937295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:45.937295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:46.010433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:46.011247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:46.011389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:46.011672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:46.012643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:46.013134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:46.013348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:46.014325 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:46.015928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:46.016554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:46.017092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:46.017475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:46.018034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:46.018485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:46.018485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:50.467739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:50.468553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:50.468626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:50.469560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:50.518493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:50.518801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.519488 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.520581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.521259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:50.521865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:50.522674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.523478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:50.523478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.523478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.523478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:50.523478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.523478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:50.658823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:50.659297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:50.659388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:50.660359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:50.660703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.661561 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.662557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.663169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.663805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:50.664593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.665541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.665541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:50.665541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.665541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:50.779864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:50.780535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:50.780717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:50.780931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:50.782311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:50.782726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.783588 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:50.785177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.786208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.787085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:50.787824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.788401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.788401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:50.788401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.788401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.789050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:50.789659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.790240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.790240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:50.790240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.790240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:50.927590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:50.928186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:50.928328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:50.928488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:50.929426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:50.929845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.930764 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:50.932065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.932843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.933536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.934170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.935067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:50.935067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.935067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:50.935316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:50.936144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:50.937203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:50.937203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:50.937203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:50.937203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:51.011274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:51.011700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:51.011905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:51.012267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:51.013119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:51.013598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:51.013842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:51.014745 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:51.016363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:51.016984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:51.017602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:51.017962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:51.018855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:51.019783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:51.019783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:55.468831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:55.469793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:55.469793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:55.471363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:55.509121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:55.509606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.510680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.511799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.512464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:55.513115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:55.513826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.514708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:55.514708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.514708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.514708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:55.514708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.514708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:55.659640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:55.660264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:55.660409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:55.661408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:55.661772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.662737 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.663876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.664170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:55.664436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.664436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.664949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.665319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:55.665688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.665688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:55.780578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:55.781049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:55.781231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:55.781376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:28:55.782231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:55.782585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.783508 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:55.784807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.785656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.786110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:55.786705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.787337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.787337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:55.787337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.787337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.787673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:55.788497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.789403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.789403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:55.789403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.789403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:55.928376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:55.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:55.929142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:55.929296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:55.929984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:55.930382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.931193 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:28:55.932595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.933412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.933942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.934624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.935245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:55.935245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.935245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:55.935641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:55.936465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:55.937027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:55.937027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:55.937027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:55.937027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:28:56.012177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:28:56.012727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:28:56.012844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:28:56.013099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:28:56.013745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:56.014151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:28:56.014350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:56.015165 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:56.016683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:28:56.017388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:56.017954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:28:56.018247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:28:56.018845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:28:56.019368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:28:56.019368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:00.469693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:00.470648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:00.470648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:00.471774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:00.506418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:00.507021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.508166 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.509604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.510281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:00.510821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:00.511487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.512266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:00.512266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.512266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.512266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:00.512266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.512266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:00.660553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:00.660908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:00.661206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:00.661980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:00.662295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.663057 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.664381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.664901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:00.665577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.665577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.666092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.666847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:00.667684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.667684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:00.781210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:00.781840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:00.781982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:00.782167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:00.783015 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:00.783428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.784316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:00.785717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.786166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.787138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:00.787528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.787898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.787898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:00.787898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.787898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.788728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:00.789307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.789870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.789870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:00.789870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.789870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:00.929144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:00.929744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:00.929880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:00.930082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:00.930819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:00.931240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.932319 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:00.933768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.934519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.935148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.935746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.936459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:00.936459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.936459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:00.936780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:00.937543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:00.938406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:00.938406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:00.938406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:00.938406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:01.012848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:01.013204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:01.013333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:01.013606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:01.014263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:01.014674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:01.014893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:01.015740 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:01.017358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:01.017935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:01.018515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:01.018912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:01.019709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:01.020656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:01.020656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:05.494122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:05.507878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:05.508329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:05.509534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:05.541642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:05.541936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.542608 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:05.543502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:05.544226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:05.544970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:05.545944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:05.546481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:05.546481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:05.546481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:05.546481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:05.546481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.546481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:05.661241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:05.668546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:05.668676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:05.781960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:05.790447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:05.790773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:05.791245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:05.929683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:05.938723 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:05.938907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:05.939036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:05.994743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:05.995076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.996984 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.328 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:05.998896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:05.999293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:06.000039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.000039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:06.000039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.001875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.002460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:06.002973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:06.004102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:06.004622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.213 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.006369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:06.006759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.007168 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.008483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:06.009037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.009566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.009566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:06.009566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.009566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:06.010141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.010603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.071 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:06.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.011958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.012362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:06.012362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.012362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.013401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:06.014407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:06.015113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:06.015483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:06.015667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.016430 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.017304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:06.017696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.018076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:06.018228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:06.018758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:06.019335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:06.019335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:10.507820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:10.508615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:10.508615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:10.509899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:10.543101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:10.543496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.544422 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.545440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.546307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:10.546825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:10.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.548324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:10.548324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.548324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.548324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:10.548324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.548324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:10.668784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:10.669348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:10.669552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:10.670242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:10.670516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.671276 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.672902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.673527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:10.674912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.674912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.675159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.675874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:10.676680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.676680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:10.790626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:10.791190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:10.791367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:10.791517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:10.792298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:10.792636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.793489 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:10.795596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.796436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.796958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:10.797726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.798356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:10.799059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.799876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:10.938715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:10.939314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:10.939464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:10.939621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:10.940210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:10.940643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.941608 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:10.943139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.943916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.944739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.945365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.945971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:10.945971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.945971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:10.946141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:10.946914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:10.949289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:10.949289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:10.949289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:10.949289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:11.014553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:11.015082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:11.015236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:11.015519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:11.016236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:11.016752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:11.016945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:11.017698 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:11.019207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:11.019815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:11.020433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:11.021120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:11.022068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:11.022558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:11.022558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:15.508576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:15.509445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:15.509445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:15.510638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:15.545420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:15.545817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.546750 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.547856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.548759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:15.549931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:15.550932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.552288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:15.552288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.552288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.552288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:15.552288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.552288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:15.669466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:15.669950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:15.670152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:15.670952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:15.671301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.672232 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.673741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:15.674910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.674910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.675450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.676493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:15.677346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.677346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:15.791610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:15.792130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:15.792278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:15.792492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:15.793347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:15.793704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.794640 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:15.795892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.796791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.797203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:15.797846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:15.799550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.800479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:15.939440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:15.939836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:15.939954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:15.940061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:15.940882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:15.941504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.942303 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:15.943581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.944532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.945268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.945839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.946498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:15.946498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.946498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:15.947260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:15.948108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:15.948957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:15.948957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:15.948957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:15.948957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:16.014970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:16.015491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:16.015658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:16.015943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:16.016493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:16.016793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:16.016982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:16.017611 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:16.019023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:16.019564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:16.020312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:16.020685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:16.021626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:16.022430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:16.022430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:20.509096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:20.509929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:20.509929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:20.510378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:20.543530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:20.543904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.544748 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.545790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.546460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:20.547996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.548890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:20.548890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.548890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.548890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:20.548890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.548890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:20.670070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:20.670993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:20.670993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:20.671522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:20.671783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.672492 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.673544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.674183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.674777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:20.675603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.676665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.676665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:20.676665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.676665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:20.792196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:20.792795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:20.792938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:20.793122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:20.793780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:20.794132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.794904 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:20.796223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.797026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.797652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:20.798285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.798901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.798901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:20.798901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.798901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.799213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:20.800203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.801188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.801188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:20.801188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.801188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:20.940114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:20.941213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:20.941213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:20.941213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:20.942158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:20.942679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.944145 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:20.945051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.945777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.946560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.947182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.947787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:20.947787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.947787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:20.948752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:20.949529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:20.951134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:20.951134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:20.951134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:20.951134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:21.015703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:21.016264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:21.016414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:21.016671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:21.017375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:21.017755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:21.017939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:21.018709 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:21.020235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:21.020830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:21.021383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:21.021957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:21.023269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:21.024458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:21.024458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:25.509744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:25.510560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:25.510560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:25.511879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:25.543516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:25.543878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.544681 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.545588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.546337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:25.547092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:25.547937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.548964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:25.548964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.548964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.548964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:25.548964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.548964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:25.670943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:25.671715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:25.671862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:25.672943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:25.673313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.674145 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.675766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.676449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.677120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:25.677879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.678979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.678979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:25.678979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.678979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:25.793089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:25.794333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:25.794333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:25.794333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:25.795364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:25.795773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.797783 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:25.798311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.798650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.799921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:25.800573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.801162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.801162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:25.801162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.801162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.801459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:25.802174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.802960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.802960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:25.802960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.802960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:25.940744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:25.941080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:25.941197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:25.941367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:25.942334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:25.942704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.943482 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:25.944831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.945586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.946341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.947050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.947626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:25.947626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.947626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:25.948141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:25.948857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:25.949338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:25.949338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:25.949338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:25.949338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:26.016227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:26.016548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:26.016684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:26.016883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:26.017752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:26.018381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:26.018381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:26.019130 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:26.020616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:26.021293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:26.021959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:26.022427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:26.023243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:26.024078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:26.024078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:30.510466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:30.511275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:30.511420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:30.512688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:30.545667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:30.546082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.546977 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.548056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.548943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:30.549756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:30.550727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.551737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:30.551737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.551737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.551737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:30.551737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.551737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:30.671570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:30.673405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:30.673405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:30.673637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:30.674075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:30.679980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.680750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:30.681434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:30.682084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.682898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:30.794062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:30.794473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:30.794645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:30.794810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:30.795492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:30.795879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.796777 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:30.798064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.799076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.799477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:30.800104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.800744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.800744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:30.800744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.800744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.801128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:30.801954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.802908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.802908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:30.802908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.802908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:30.941437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:30.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:30.941911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:30.942131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:30.943040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:30.943487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.944638 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:30.945846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.946744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.947312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.947935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.948628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:30.948628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.948628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:30.949184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:30.950072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:30.951057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:30.951057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:30.951057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:30.951057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:31.016716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:31.017510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:31.017826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:31.018166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:31.018894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:31.019403 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:31.019667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:31.021174 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:31.022531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:31.023165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:31.023547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:31.023794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:31.024335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:31.024875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:31.024875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:35.511818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:35.513379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:35.513614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:35.516314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:35.519199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=226 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:29:35.550923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:35.551502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.552941 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.553621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:35.554980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:35.555814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.556843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:35.556843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.556843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.556843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:35.556843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.556843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:35.672443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:35.673043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:35.673212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:35.674922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:35.675320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.677274 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.678494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.678870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:35.679283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.679283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.679360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.679951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:35.680588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.680588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:35.794734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:35.795415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:35.795502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:35.795635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:35.796464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:35.796819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.797703 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:35.798813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.799598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:35.800246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:35.801423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.802260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:35.942389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:35.942840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:35.942992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:35.943188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:35.943733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:35.944268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.945084 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:35.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.947225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.947709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.948294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.948892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:35.948892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.948892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:35.949490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:35.950322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:35.951124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:35.951124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:35.951124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:35.951124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:36.017586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:36.018044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:36.018237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:36.018466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:36.018938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:36.019329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:36.019512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:36.020170 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:36.021452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:36.022105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:36.022675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:36.023089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:36.023854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:36.024702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:36.024702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:40.513551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:40.514368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:40.514521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:40.515976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:40.554718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:40.554915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.556119 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.557404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.557977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:40.558520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.558520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:40.558621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.559326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.560129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:40.560129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.560129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:40.673110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:40.673378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:40.673448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:40.674255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:40.674677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.675306 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.676563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.677209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:40.677827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.677827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.678455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.678946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:40.679479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.679479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:40.795353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:40.795818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:40.795921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:40.796010 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:40.796746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:40.796887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.797545 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:40.798485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.798892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.799307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:40.799606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.799963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.799963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:40.799963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.799963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:40.800715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.801183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.801183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:40.801183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.801183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:40.942940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:40.943386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:40.943463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:40.943524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:40.944249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:40.944582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.945326 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:40.946016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.946745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:40.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:40.947956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:40.949404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:41.018179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:41.019085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:41.019085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:41.019327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:41.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:41.020173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:41.020370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:41.021229 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:41.022686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:41.023372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:41.024165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:41.024409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:41.025246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:41.025796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:41.025796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:45.514173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:45.514833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:45.514833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:45.515369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:45.548438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:45.549702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.550046 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.551124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.551773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:45.552377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:45.553164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.554144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:45.554144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.554144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.554144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:45.554144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.554144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:45.673888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:45.674334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:45.674466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:45.675375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:45.675642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.676969 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.679470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:45.680046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.680046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.681115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.681755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:45.682287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.682287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:45.796245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:45.796675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:45.796818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:45.796966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:45.797701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:45.798060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.798881 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:45.800185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.800920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.801530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:45.802151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.802779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.802779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:45.802779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.802779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.803222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:45.803957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.804977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.804977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:45.804977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.804977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:45.943668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:45.944130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:45.944280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:45.944446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:45.944933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:45.945344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.946046 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:45.947317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.948767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.949359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.950061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:45.950061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.950061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:45.950581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:45.951354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:45.952131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:45.952131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:45.952131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:45.952131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:46.019178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:46.020031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:46.020031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:46.020031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:46.020506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:46.021217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:46.021217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:46.022140 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:46.023395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:46.024042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:46.024607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:46.025408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:46.026398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:46.026398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:46.026398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:50.514757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:50.515451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:50.515579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:50.516883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:50.553905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:50.554313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.555201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.556281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.556848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:50.557388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.557388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:50.557646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.558369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.559174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:50.559174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.559174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:50.674406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:50.674944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:50.675126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:50.675923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:50.676284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.677660 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.678666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.679266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:50.679789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.679789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.680288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.681097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:50.681876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.681876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:50.796935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:50.797516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:50.797682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:50.797807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:50.798466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:50.798800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.799644 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:50.801065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.801959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.802634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:50.803285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.803885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.803885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:50.803885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.803885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.804248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:50.804795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.805298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.805298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:50.805298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.805298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:50.944486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:50.945065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:50.945288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:50.945534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:50.946201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:50.946624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.947570 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:50.948991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.949742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.950396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.950902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.951591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:50.951591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.951591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:50.952115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:50.952856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:50.953647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:50.953647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:50.953647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:50.953647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:51.019809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:51.020107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:51.020253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:51.020481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:51.020951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:51.021337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:51.021529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:51.022322 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:51.023816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:51.024571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:51.025402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:51.025638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:51.026425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:51.027222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:51.027222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:55.515555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:55.516895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:55.517123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:55.518444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:55.552505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:55.552850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.553765 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.554965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.555686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:55.556672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:55.557534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.558221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:55.558221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.558221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.558221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:55.558221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.558221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:55.675149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:55.675943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:55.676138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:55.677787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:55.678236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.679248 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.681195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.681484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:55.681949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.681949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.682669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.683060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:55.683768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.683768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:55.797737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:55.801023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:55.801023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:55.801023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:29:55.801552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:55.801784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.802420 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:55.803386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.803766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.804196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:55.804511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.804793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.804793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:55.804793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.804793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.807695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:55.808098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.808506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.808506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:55.808506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.808506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:55.945215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:55.945799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:55.945888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:55.945970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:55.946679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:55.947195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.948066 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:29:55.948836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.949489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:55.949928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:55.950404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.950811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:55.951395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:29:56.020307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:29:56.021046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:29:56.021376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:29:56.021822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:29:56.022579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:56.022964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:29:56.023220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:56.024085 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:56.025496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:29:56.026096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:56.026648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:29:56.027288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:29:56.028131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:29:56.028976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:29:56.028976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:00.517193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:00.518222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:00.518439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:00.519699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:00.552567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:00.553113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.554301 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.555576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.556208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:00.556831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.556831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:00.557134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.557892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.558916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:00.558916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.558916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:00.676188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:00.676960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:00.677106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:00.678149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:00.678472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.679343 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.681293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.681860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:00.682503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.682503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.682982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.683750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:00.684640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.684640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:00.798640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:00.799267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:00.799369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:00.799549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:00.801274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:00.801274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.802420 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:00.802829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.803546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.804280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:00.804812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.805197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.805197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:00.805197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.805197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.805917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:00.806381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.806898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.806898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:00.806898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.806898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:00.946148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:00.946750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:00.946903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:00.947103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:00.948474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:00.948996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.950232 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:00.951636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.952388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.953050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.953616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.954262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:00.954262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.954262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:00.954725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:00.955460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:00.956271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:00.956271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:00.956271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:00.956271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:01.021151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:01.021784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:01.021946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:01.022235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:01.023022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:01.023535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:01.023773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:01.024785 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:01.026383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:01.026908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:01.027459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:01.028115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:01.028811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:01.029659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:01.029659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:05.518027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:05.519146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:05.519146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:05.520069 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:05.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:05.550030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.550875 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.551681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.552334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:05.552914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:05.553657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.554532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:05.554532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.554532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.554532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:05.554532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.554532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:05.677037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:05.677599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:05.677752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:05.678656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:05.678972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.679810 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.681299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.681885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:05.682502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.682502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.683051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.684537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:05.685480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.685480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:05.799464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:05.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:05.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:05.800307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:05.800828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:05.801278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.801846 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:05.803212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.804124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.804655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:05.805778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.807066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:05.808296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.809328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:05.946872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:05.947311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:05.947498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:05.947636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:05.948258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:05.948647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.949401 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:05.950614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.951405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.952270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.952898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.953541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:05.953541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.953541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:05.953876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:05.955398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:05.956945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:05.956945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:05.956945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:05.956945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:06.021808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:06.022247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:06.022398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:06.022622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:06.023151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:06.023598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:06.023792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:06.024543 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:06.025899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:06.026540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:06.027137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:06.027511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:06.028493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:06.029325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:06.029325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:10.518816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:10.519693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:10.519891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:10.521195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:10.566160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:10.566434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.567655 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.569194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.569447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:10.569689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.569689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:10.571523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.571941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.572382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:10.572382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.572382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:10.677772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:10.678262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:10.678418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:10.679405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:10.679684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.680605 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.682190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.682803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:10.683482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.683482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.683971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.684771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:10.685658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.685658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:10.800047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:10.800587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:10.800743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:10.800908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:10.801719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:10.802103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.802935 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:10.805578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.806301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:10.806937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.807575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.808097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:10.808676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.809231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.809231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:10.809231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.809231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:10.947733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:10.948384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:10.948553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:10.948751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:10.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:10.950192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.951522 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:10.954129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.954973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.955599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.956198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.956890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:10.956890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.956890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:10.957288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:10.958179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:10.959046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:10.959046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:10.959046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:10.959046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:11.022452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:11.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:11.023248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:11.023568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:11.024218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:11.024500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:11.024621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:11.025579 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:11.027436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:11.028365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:11.029385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:11.030036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:11.030931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:11.031901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:11.031901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:15.519592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:15.520325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:15.520325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:15.521622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:15.556291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:15.556291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.556537 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.557774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.558465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:15.559100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.559100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:15.559751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.560582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.561478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:15.561478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.561478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:15.678246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:15.678635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:15.678769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:15.680795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:15.680795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.680991 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.682520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.683143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:15.683531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.683531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.684493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.684945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:15.685472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.685472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:15.800793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:15.801277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:15.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:15.801448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:15.802906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:15.802906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.803185 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:15.804496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.805194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.805754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:15.806314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.806942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.806942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:15.806942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.806942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.807754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:15.808608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.809500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.809500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:15.809500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.809500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:15.948932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:15.949519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:15.949680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:15.949838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:15.951592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:15.951592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.951799 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:15.953305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.954070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.954785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.955327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.955716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:15.955716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.955716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:15.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:15.957043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:15.957577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:15.957577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:15.957577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:15.957577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:16.023253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:16.023881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:16.024029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:16.024308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:16.024841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:16.025392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:16.025763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:16.026684 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:16.027802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:16.028488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:16.029046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:16.029838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:16.030719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:16.030719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:16.030719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:20.520274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:20.521062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:20.521170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:20.522432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:20.552095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:20.553558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.553800 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.554727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.555383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:20.556025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.556025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:20.556456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.557516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.558549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:20.558549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.558549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:20.681119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:20.681119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:20.681119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:20.681817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:20.682189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.682851 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.683780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.684166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:20.684529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.684529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.684839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.685327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:20.685792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.685792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:20.801501 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:20.803100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:20.803100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:20.803100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:20.804953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:20.804953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.805295 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:20.807434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.808219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.809820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.810631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.810631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:20.810631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.810631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.811871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:20.812783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.813726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.813726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:20.813726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.813726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:20.949643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:20.950367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:20.950519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:20.950716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:20.952715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:20.952715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.952961 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:20.954445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.955191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.955818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.956555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.956939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:20.956939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.956939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:20.957469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:20.958335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:20.958862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:20.958862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:20.958862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:20.958862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:21.024128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:21.024525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:21.024618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:21.024791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:21.026802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:21.026802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:21.026802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:21.027046 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:21.028026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:21.028639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:21.029223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:21.030525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:21.031540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:21.032634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:21.032634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:25.520965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:25.521714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:25.521714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:25.522812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:25.558129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:25.558722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.559807 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.560939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.561586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:25.562351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:25.563018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.563869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:25.563869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.563869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.563869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:25.563869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.563869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:25.679693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:25.679937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:25.680094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:25.681144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:25.681460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.682394 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.683764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.684365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:25.684897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.684897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.685309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.686088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:25.686882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.686882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:25.802198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:25.802631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:25.802770 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:25.802910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:25.803700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:25.804107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.804924 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:25.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.806963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.807499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:25.808068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.808633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.808633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:25.808633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.808633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.809145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:25.809955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.810934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.810934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:25.810934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.810934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:25.950334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:25.950865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:25.950998 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:25.951274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:25.951937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:25.952538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.953474 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:25.954693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.955625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.956147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.956841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.957780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:25.957780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.957780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:25.957972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:25.959140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:25.960323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:25.960323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:25.960323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:25.960323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:26.024748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:26.025155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:26.025260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:26.025519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:26.026208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:26.026669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:26.026917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:26.027686 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:26.029068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:26.029870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:26.030617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:26.031219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:26.031866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:26.031866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:26.031866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:30.521714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:30.522468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:30.522468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:30.523621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:30.555634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:30.556073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.557086 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.558728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.559303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:30.559829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.559829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:30.560465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.561158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.561940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:30.561940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.561940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:30.680216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:30.680791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:30.680955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:30.681908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:30.682271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.683127 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.684927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.685558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:30.686132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.686132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.686646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.687461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:30.688220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.688220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:30.802790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:30.803311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:30.803424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:30.803581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:30.804173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:30.804456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.805181 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:30.806529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.806945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.807869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:30.808253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.809377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.809377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:30.809377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.809377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.810157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:30.810615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.811158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.811158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:30.811158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.811158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:30.950968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:30.951351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:30.951430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:30.951654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:30.952240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:30.952660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.953215 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:30.954163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.955139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:30.955901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:30.956768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.958511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:30.959503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:31.025414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:31.025801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:31.025962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:31.026259 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:31.026774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:31.027169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:31.027346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:31.028160 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:31.029783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:31.030488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:31.031097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:31.031528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:31.032387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:31.033377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:31.033377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:35.522479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:35.523320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:35.523409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:35.524805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:35.526824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:30:35.557686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:35.557940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.558646 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.559740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.560402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:35.561030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:35.561739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:35.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:35.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:35.680943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:35.681560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:35.681685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:35.682735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:35.682937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.683610 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.685160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.685815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:35.686504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.686504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.686781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.688573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:35.690048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.690048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:35.803496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:35.803900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:35.804064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:35.804212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:35.804852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:35.805083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.805661 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:35.806852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.807746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.808137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:35.808689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.809349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.809349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:35.809349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.809349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.809995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:35.810517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.811072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.811072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:35.811072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.811072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:35.951666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:35.952215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:35.952350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:35.952507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:35.953270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:35.953522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.954083 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:35.955289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.955956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.956601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.957267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.957880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:35.957880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.957880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:35.958124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:35.958868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:35.959698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:35.959698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:35.959698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:35.959698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:36.026022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:36.026568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:36.026645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:36.026807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:36.027165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:36.027393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:36.027483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:36.027958 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:36.028707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:36.029376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:36.029909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:36.030655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:36.031560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:36.031560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:36.031560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:40.523138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:40.523890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:40.523890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:40.525459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:40.557175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:40.557604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.559339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.561371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:40.561977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:40.562752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.563651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:40.563651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.563651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.563651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:40.563651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.563651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:40.681716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:40.682200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:40.682486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:40.683683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:40.684027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.684986 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.686672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.687403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:40.687914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.687914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.689307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.690057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:40.690571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.690571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:40.804909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:40.805465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:40.805694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:40.805978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:40.806974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:40.807427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.808315 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:40.809269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.809994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.810479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:40.811148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.811751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.811751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:40.811751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.811751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.812989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:40.813507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.814127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.814127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:40.814127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.814127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:40.952552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:40.953257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:40.953403 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:40.953568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:40.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:40.954720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.955548 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:40.956914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.957634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.958304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.958857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.959477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:40.959477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.959477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:40.959964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:40.960749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:40.961541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:40.961541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:40.961541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:40.961541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:41.026986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:41.027677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:41.027831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:41.027966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:41.028518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:41.028893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:41.029111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:41.029851 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:41.031138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:41.031766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:41.032186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:41.032709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:41.033194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:41.033722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:41.033722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:45.523788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:45.524589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:45.524589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:45.525798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:45.558679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:45.559171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.560167 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.561031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.561676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:45.562346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:45.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.564144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:45.564144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.564144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.564144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:45.564144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.564144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:45.682361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:45.682918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:45.683075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:45.683928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:45.684298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.684966 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.686548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:45.687699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.687699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.688371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.689143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:45.689952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.689952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:45.805372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:45.805750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:45.805887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:45.806064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:45.806672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:45.806889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.807510 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:45.808620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.809552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:45.810489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.811226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:45.812176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.813240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:45.953323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:45.954359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:45.954359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:45.954359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:45.954818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:45.955269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.956406 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:45.957398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.958102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.958800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.959354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.959946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:45.959946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.959946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:45.960599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:45.961440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:45.962250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:45.962250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:45.962250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:45.962250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:46.027664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:46.028074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:46.028218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:46.028456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:46.028912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:46.029333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:46.029553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:46.030433 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:46.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:46.032471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:46.033091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:46.033383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:46.034238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:46.035113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:46.035113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:50.524414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:50.525079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:50.525380 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:50.526757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:50.559787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:50.560247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.561223 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.562293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.562828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:50.563692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:50.564361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.564819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:50.564819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.564819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.564819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:50.564819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.564819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:50.682949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:50.683645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:50.683815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:50.684737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:50.685222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.686358 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.687715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.688408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:50.689040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.689040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.690431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.691601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:50.693450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.693450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:50.805929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:50.806503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:50.806647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:50.806797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:50.807640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:50.808223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.809177 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:50.810868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.811877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.812320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:50.813035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.813714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.813714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:50.813714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.813714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.813935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:50.814791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:50.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:50.954117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:50.954606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:50.954690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:50.954841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:50.955607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:50.956022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.956894 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:50.958243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.958995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.959611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.960246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.960872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:50.960872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.960872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:50.961256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:50.962093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:50.962657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:50.962657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:50.962657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:50.962657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:51.028348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:51.028733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:51.028896 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:51.029143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:51.029707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:51.030157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:51.030376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:51.031134 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:51.032618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:51.033249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:51.033798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:51.034224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:51.035159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:51.036196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:51.036196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:51.381316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=d1b8ed5c-a733-445b-b843-b8cff2dd0af7, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a'}, gateway_chassis=[d25cc42b-d940-49eb-ba36-9dc2670d10dc], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:92:a4:24', name='lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b', networks=['172.24.5.236/24', '2001:db8::2c9/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=107007) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 13 02:30:53.855107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=b1631d63-c41d-43c8-8baf-1ad8660b6fa1, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f', 'neutron:subnet_ids': '97900556-2eeb-4240-89bd-ed8258b9022d'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:d9:96:58', name='lrp-b4b257f2-5fb2-416d-8a65-4b79cca24999', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=107007) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 13 02:30:53.881113 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe synchronized; highest priority chassis 75e1c21d-535b-42ed-8236-406877c7afe8
Apr 13 02:30:53.881798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'neutron:router_id': '530b037d-a47e-4f63-bd55-041139eeee2f'}}) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:53.882185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, chassis=75e1c21d-535b-42ed-8236-406877c7afe8, priority=1, columns={}) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:53.882401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:53.882973 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2410, in link_network_ha_chassis_group
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     next(self.gen)
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 13 02:30:53.883959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise result.ex
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:53.884935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.event 
Apr 13 02:30:55.525203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:55.525841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:55.525976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:55.527532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:55.567200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:55.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.568584 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.569623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.570203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:55.570690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:55.571348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.572088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:55.572088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.572088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.572088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:55.572088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.572088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:55.683853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:55.684233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:55.684388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:55.685386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:55.685721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:55.685948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.686799 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.688908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.689453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:55.689939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.689939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.690592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.691342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:55.692086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.692086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:55.806657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:55.807631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:55.807783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:55.807925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:30:55.808742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:55.809137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.810206 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:55.811850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.812525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.812966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:55.813394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.813868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:55.814485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.816034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:55.954821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:55.955427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:55.955568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:55.955752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:55.956354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:55.956645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.957463 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:30:55.958753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.959454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.960271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.961536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:55.962864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:55.964102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:55.965099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:30:56.028895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:30:56.029243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:30:56.029470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:30:56.029727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:30:56.032032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:56.032423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:30:56.032809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:56.033746 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:56.035941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:30:56.036741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:56.037418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:30:56.038794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:30:56.040216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:30:56.041953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:30:56.041953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:00.526238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:00.527430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:00.527430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:00.528683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:00.564841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:00.565333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.566508 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.567750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.568146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:00.568470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.568470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:00.568861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.569366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.569877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:00.569877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.569877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:00.684788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:00.684788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:00.684788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:00.686358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:00.686847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:00.687166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.688275 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.689977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.690709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:00.691362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.691362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.691735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.692627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:00.693682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.693682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:00.807288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:00.807606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:00.807745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:00.807912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:00.809148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:00.809584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.810733 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:00.812035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.812801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.813286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:00.813979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.814630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.814630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:00.814630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.814630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.814854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:00.815633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.816548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.816548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:00.816548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.816548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:00.955579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:00.955769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:00.955876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:00.956088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:00.957341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:00.957824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.958882 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:00.960186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.960922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.961524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.962259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.963044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:00.963044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.963044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:00.963241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:00.963964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:00.964865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:00.964865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:00.964865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:00.964865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:01.029493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:01.029693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:01.029822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:01.030028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:01.030883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:01.031436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:01.031702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:01.032733 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:01.034143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:01.034943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:01.035596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:01.035680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:01.036438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:01.037328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:01.037328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:05.526953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:05.527743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:05.527743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:05.528952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:05.563443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:05.563832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.564735 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.565610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.566297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:05.566859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:05.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.568347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:05.568347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.568347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.568347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:05.568347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.568347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:05.685036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:05.685777 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:05.685935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:05.687243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:05.687602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:05.687839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.688755 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.690576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.691139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:05.691684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.691684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.692358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.693112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:05.693899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.693899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:05.807947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:05.808315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:05.808451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:05.808611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:05.809536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:05.809883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.810784 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:05.812121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.812831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.813376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:05.813973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.814629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.814629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:05.814629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.814629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.814906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:05.815767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.816697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.816697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:05.816697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.816697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:05.956043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:05.956623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:05.956767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:05.956932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:05.957721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:05.958151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.959019 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:05.960324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.961079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.961691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.962359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.962951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:05.962951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.962951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:05.963378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:05.964175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:05.964992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:05.964992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:05.964992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:05.964992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:06.029957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:06.030257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:06.030376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:06.030591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:06.031349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:06.031762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:06.032026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:06.032916 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:06.034361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:06.034961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:06.035527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:06.035817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:06.036613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:06.037448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:06.037448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:10.527577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:10.528613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:10.528703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:10.530905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:10.570585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:10.571141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.572108 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.574573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.575292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:10.575920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.575920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:10.576843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.577649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.578568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:10.578568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.578568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:10.685593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:10.686225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:10.686347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:10.687605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:10.688058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:10.688338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.689353 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.691306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.692096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:10.692853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.692853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.693616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:10.694185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.694185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:10.808616 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:10.809187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:10.809353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:10.809512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:10.810276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:10.810661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.811541 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:10.812923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.813420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.814272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:10.814662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.815059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.815059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:10.815059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.815059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.815907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:10.816477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.817113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.817113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:10.817113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.817113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:10.956740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:10.957559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:10.957659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:10.957817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:10.958795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:10.959300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.960302 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:10.961401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.962297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:10.963131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:10.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.964617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:10.965498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:11.030471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:11.030793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:11.030910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:11.031220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:11.031720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:11.032105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:11.032360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:11.033133 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:11.034736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:11.035547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:11.036217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:11.036345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:11.037201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:11.038079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:11.038079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:15.528397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:15.529198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:15.529436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:15.530666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:15.571078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:15.571266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.571910 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.572645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.573219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:15.573782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:15.574455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.575270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:15.575270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.575270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.575270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:15.575270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.575270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:15.686508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:15.687309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:15.687502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:15.688774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:15.689163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:15.689405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.690316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.691527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.692163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.692882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:15.693712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.694804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.694804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:15.694804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.694804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:15.809497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:15.810139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:15.810272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:15.810444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:15.811418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:15.811758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.812886 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:15.814207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.815547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.815937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:15.816568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.817289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.817289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:15.817289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.817289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.817490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:15.818300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.819365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.819365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:15.819365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.819365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:15.959148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:15.960028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:15.960304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:15.960615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:15.961410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:15.961860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.962801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:15.964148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.964836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.965603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.966163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.966759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:15.966759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.966759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:15.967990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:15.968774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:15.969758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:15.969758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:15.969758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:15.969758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:16.031352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:16.031877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:16.032116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:16.032480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:16.033925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:16.033925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:16.033925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:16.034664 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:16.036696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:16.037164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:16.037569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:16.037979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:16.038501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:16.039149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:16.039149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:20.529628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:20.530421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:20.530605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:20.532038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:20.569139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:20.569639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.570547 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.571967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.572575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:20.573166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.573166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:20.573731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.574475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.575130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:20.575130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.575130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:20.687500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:20.688246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:20.688397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:20.689664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:20.690026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:20.690294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.691177 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.692944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.693268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:20.693652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.693652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.693990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.694474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:20.694986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.694986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:20.810237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:20.810773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:20.810892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:20.811072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:20.811670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:20.812026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.812778 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:20.813925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.814908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.815309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:20.815920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.817122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:20.817517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.818041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:20.959970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:20.961551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:20.961645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:20.961718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:20.962100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:20.962338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.962831 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:20.963457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.964217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:20.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:20.965481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.966069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:20.966898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:21.032748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:21.033144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:21.033231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:21.033480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:21.034145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:21.034394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:21.034519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:21.035114 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:21.036517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:21.037124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:21.037683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:21.038031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:21.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:21.039601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:21.039601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:25.530243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:25.531233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:25.531233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:25.532752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:25.572302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:25.572700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.573579 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.574550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.575190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:25.575918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:25.576700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.577738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:25.577738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.577738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.577738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:25.577738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.577738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:25.688435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:25.688870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:25.689025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:25.690177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:25.690517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:25.690760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.691632 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.693110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.693803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:25.694490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.694490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.694645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.695454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:25.695923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.695923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:25.810866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:25.811397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:25.811584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:25.811688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:25.812161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:25.812401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.812981 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:25.814317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.815198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.815759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:25.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.817070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.817070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:25.817070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.817070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.817519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:25.818419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.819398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.819398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:25.819398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.819398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:25.961751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:25.962336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:25.962509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:25.962645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:25.963177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:25.963573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.964348 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:25.965639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.966422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.966928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.967520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:25.968157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:25.968862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:25.969715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:26.033313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:26.033676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:26.033676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:26.034013 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:26.034910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:26.035618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:26.035618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:26.036399 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:26.037538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:26.038238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:26.038913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:26.039261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:26.040461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:26.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:26.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:30.534091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:30.534091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:30.534091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:30.534091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:30.572850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:30.573396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.574375 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.576454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.576727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:30.576968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.576968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:30.577575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.578067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.578523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:30.578523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.578523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:30.688884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:30.690411 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:30.690589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:30.691761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:30.692164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:30.692442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.693818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.694999 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.695612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:30.696158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.696158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:30.698892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.698892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:30.811565 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:30.812318 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:30.812396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:30.812601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:30.813424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:30.813830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.814802 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:30.815773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.816490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.816835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:30.817385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.817948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:30.818675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.819494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:30.962382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:30.962886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:30.962983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:30.963172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:30.963628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:30.964133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.964741 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:30.965888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.966585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.967399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.967936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.968531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:30.968531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.968531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:30.969594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:30.970334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:30.970799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:30.970799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:30.970799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:30.970799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:31.034081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:31.035443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:31.035443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:31.035443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:31.035985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:31.036493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:31.036660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:31.040929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:31.041770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:31.042148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:31.042523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:31.042982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:35.531456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:35.532619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:35.532811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:35.534798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:35.537323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=257 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:31:35.570311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:35.570571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.571303 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.572092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:35.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:35.574290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.575323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:35.575323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.575323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.575323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:35.575323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.575323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:35.689614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:35.689914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:35.689914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:35.690741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:35.691032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:35.691142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.691689 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.692583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.693278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.693847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:35.694652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.695711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.695711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:35.695711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.695711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:35.812214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:35.812536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:35.812629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:35.812789 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:35.813310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:35.813728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.814570 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:35.815821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.816582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.817152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:35.817713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.818286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.818286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:35.818286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.818286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.818873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:35.819682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.820570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.820570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:35.820570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.820570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:35.963105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:35.963656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:35.963788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:35.963948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:35.964572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:35.964970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.965756 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:35.967074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.967784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.968419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.969057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.969796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:35.969796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.969796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:35.970190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:35.971132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:35.971921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:35.971921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:35.971921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:35.971921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:36.034941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:36.035217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:36.035307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:36.035549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:36.035938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:36.036320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:36.036544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:36.037251 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:36.038629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:36.039313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:36.039880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:36.040022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:36.040801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:36.041545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:36.041545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:40.532248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:40.533313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:40.533313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:40.533908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:40.572331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:40.572637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.573585 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.574708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.575594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:40.576212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:40.576825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.577736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:40.577736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.577736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.577736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:40.577736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.577736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:40.690276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:40.690564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:40.690691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:40.691809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:40.692126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:40.692361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.693231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.694834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.695488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:40.696124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.696124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.696311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.697167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:40.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.698088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:40.814269 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:40.814816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:40.814910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:40.815130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:40.815485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:40.815788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.816394 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:40.817123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.817637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.818148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:40.818560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.819119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.819119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:40.819119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.819119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.819284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:40.819701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.820119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.820119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:40.820119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.820119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:40.963765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:40.964384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:40.964546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:40.964711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:40.965774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:40.966154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.967156 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:40.968592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.969095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.969942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.970370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.970749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:40.970749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.970749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:40.972295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:40.972760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:40.973316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:40.973316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:40.973316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:40.973316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:41.035533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:41.035985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:41.036089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:41.036253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:41.036964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:41.037495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:41.037789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:41.038671 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:41.040291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:41.041095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:41.041635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:41.041891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:41.042647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:41.043483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:41.043483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:45.492263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:45.493307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:45.493307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:45.503821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 02:31:45.504120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Number of inconsistencies found at create/update: networks=3, security_groups=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 02:31:45.504317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:31:45.533367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:45.533971 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:45.534422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:45.538242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:45.540683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.541484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.542544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.546793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Traceback (most recent call last):
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.548086 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.551596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.553915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.554751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.554751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.554751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.554751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:31:45.609566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.610139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.611269 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.076 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.613565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.614211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.614717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.614717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:45.615503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.616524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.617473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:45.617473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.617473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:45.617473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.617773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.618329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.618518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Traceback (most recent call last):
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.619495 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.620893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.621245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.621638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.621638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.621638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.621638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:31:45.652339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.652339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.652706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.652816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Traceback (most recent call last):
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.653561 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.654468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.655061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.655605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Maintenance task: Fixing resource 73a96f69-d469-4999-997b-cc72d364ae2d (type: security_groups) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:31:45.691157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:45.692965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:45.693198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:45.695788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.696192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.696491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.699442 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.701993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.702865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:45.703698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.705328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.705328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:45.705328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.705328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:45.708226 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.db.ovn_revision_numbers_db [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Successfully bumped revision number for resource 73a96f69-d469-4999-997b-cc72d364ae2d (type: security_groups) to 9
Apr 13 02:31:45.708424 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] OVN maintenance task check_for_inconsistencies finished in 0.215 seconds
Apr 13 02:31:45.861475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:45.862195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:45.862346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:45.862518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:45.863377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.863749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.864605 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:45.866078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.866489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.867547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.867833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.868143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.868143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.868143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.868143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.868900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:45.869281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.869729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.869729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:45.869729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.869729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:45.964430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:45.965142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:45.965300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:45.965482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:45.966235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:45.966547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.967424 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:45.968836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.969674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.970244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.970820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.971438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:45.971438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.971438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:45.971959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:45.972832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:45.973728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:45.973728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:45.973728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:45.973728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:46.036200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:46.036615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:46.036761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:46.036981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:46.037557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:46.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:46.038210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:46.038996 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:46.040441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:46.041275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:46.041830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:46.041924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:46.042832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:46.043698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:46.043698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:50.534531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:50.538133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:50.538133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:50.538133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:50.574051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:50.574647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:50.579040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:50.579882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.580614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:50.580614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.580614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.580614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:50.580614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.580614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:50.581344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.582439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.583681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:50.583681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.583681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:50.693448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:50.694115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:50.694255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:50.696166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:50.696473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:50.696655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.697403 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.699023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.699574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:50.700072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.700072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.701306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.701908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:50.702565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.702565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:50.863040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:50.863040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:50.863040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:50.863040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:50.863608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:50.863973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.864703 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:50.868044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.868904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:50.869696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.870168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.871393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:50.872151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.873031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.873031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:50.873031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.873031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:50.965310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:50.966115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:50.966247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:50.966413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:50.967231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:50.967601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.968442 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:50.969672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.970626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.971312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.971687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:50.972128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:50.972680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:50.973654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:51.036955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:51.037383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:51.037513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:51.037745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:51.038260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:51.038583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:51.038789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:51.039392 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:51.040712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:51.041344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:51.041771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:51.042485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:51.043641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:51.044627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:51.044627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:55.536175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:55.540063 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:55.540063 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:55.540063 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:55.573825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:55.574322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:55.577415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:55.578045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.578431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:55.578824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.579406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:55.694882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:55.695300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:55.695422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:55.696416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:55.696655 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:55.696840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.697679 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.699204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.699930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:55.700592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.700592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.701701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.702785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:55.703512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.703512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:55.863898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:55.863898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:55.863898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:55.863898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:31:55.864511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:55.864835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:55.869037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.869685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.870229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.870662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:55.871055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.871055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.871434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:55.871934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.872679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.872679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:55.872679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.872679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:55.966378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:55.966965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:55.967224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:55.967410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:55.968257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:55.968613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.969488 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:31:55.970935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.971295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.972659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.973283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:55.973283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.973283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:55.974415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:55.974810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:55.975222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:55.975222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:55.975222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:55.975222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:31:56.037672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:31:56.038283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:31:56.038402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:31:56.038693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:31:56.039280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:56.039745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:31:56.040036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:56.040911 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:56.042472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:31:56.043198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:56.043841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:31:56.044304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:31:56.045903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:31:56.047662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:31:56.047662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:00.537900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:00.538685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:00.538685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:00.540308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:00.581093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:00.581093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.581443 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.582579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.584381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:00.585285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:00.586270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.589239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:00.589239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.589239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.589239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:00.589239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.589239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:00.695500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:00.696215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:00.696436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:00.699348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:00.699348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:00.699348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.699621 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.701533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.703320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.704198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:00.705243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.705652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.705652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:00.705652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.705652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:00.863949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:00.864821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:00.864821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:00.864821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:00.867531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:00.867531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.867934 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:00.869509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.870547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:00.871475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.872365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:00.874297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.875514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:00.967349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:00.967902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:00.968137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:00.968406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:00.970462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:00.970462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.970870 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:00.972278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.975069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:00.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:00.978839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:00.979851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:01.038645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:01.039917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:01.039917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:01.039917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:01.040707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:01.041506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:01.042361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:01.042614 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:01.045594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:01.046017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:01.046447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:01.046969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:01.047682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:01.047682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:01.047682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:05.538839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:05.539716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:05.539716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:05.540873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:05.578184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:05.578628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.579653 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.580562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.581157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:05.581685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:05.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.583252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:05.583252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.583252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.583252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:05.583252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.583252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:05.696199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:05.696988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:05.697194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:05.698754 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:05.699242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:05.699522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.700546 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.702513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.703516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:05.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.704897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.705439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:05.705930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.705930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:05.864799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:05.865211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:05.865332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:05.865519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:05.866365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:05.866718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.867666 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:05.869906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.871049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.871944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:05.872566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.873294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:05.873965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.874865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:05.967931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:05.968696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:05.968872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:05.969157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:05.969891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:05.970518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.971478 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:05.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.973848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.974729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.975391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.976162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:05.976162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.976162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:05.976411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:05.977220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:05.977977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:05.977977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:05.977977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:05.977977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:06.039546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:06.040169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:06.040269 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:06.040498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:06.040943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:06.041219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:06.041388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:06.042602 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:06.043464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:06.043730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:06.044030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:06.044835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:06.045596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:06.046519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:06.046519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:10.539428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:10.540282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:10.540282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:10.540596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:10.595937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:10.596308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.597026 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.597908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.598756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:10.599581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:10.600539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.601706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:10.601706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.601706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.601706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:10.601706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.601706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:10.697098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:10.697519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:10.697627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:10.698944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:10.699361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:10.699607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.700565 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.703060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.703705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.704386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:10.705634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.706969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.706969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:10.706969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.706969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:10.865384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:10.866443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:10.866443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:10.866699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:10.867817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:10.868233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:10.872076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.873079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.873939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.874652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:10.875456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.875456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.878418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:10.878826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.879213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.879213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:10.879213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.879213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:10.968780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:10.969497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:10.969646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:10.969842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:10.970723 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:10.971019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.971734 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:10.973557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.974373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.975645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.976111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.976533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:10.976533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.976533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:10.977941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:10.978482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:10.979103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:10.979103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:10.979103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:10.979103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:11.040147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:11.040485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:11.040608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:11.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:11.041438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:11.041820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:11.042129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:11.042867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:11.044166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:11.044818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:11.045442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:11.045662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:11.046569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:11.047461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:11.047461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:15.539888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:15.540474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:15.540474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:15.540931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:15.574983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:15.575382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.576289 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.577237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.577957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:15.578607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:15.579416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.580332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:15.580332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.580332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.580332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:15.580332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.580332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:15.697792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:15.698758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:15.698899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:15.700517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:15.700942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:15.701253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.702412 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.703983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.704573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:15.705186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.705186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.705561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.706410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:15.706964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.706964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:15.866414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:15.866815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:15.866939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:15.867168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:15.868423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:15.868859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.869920 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:15.871646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.872444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.872796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:15.873412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.874136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.874136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:15.874136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.874136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.874368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:15.875125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.875940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.875940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:15.875940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.875940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:15.969484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:15.970159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:15.970305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:15.970379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:15.971227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:15.971890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.972979 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:15.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.974605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.974969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.975580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:15.976211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:15.976941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:15.977807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:16.043110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:16.043588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:16.043735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:16.044630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:16.046115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:16.046459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:16.046535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:16.047485 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:16.048926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:16.049660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:16.050435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:16.050533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:16.051547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:16.052507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:16.052507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:20.544358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:20.546631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:20.547113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:20.548604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:20.589705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:20.590109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.590992 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.592065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.592702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:20.593341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:20.594049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.594946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:20.594946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.594946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.594946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:20.594946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.594946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:20.698858 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:20.699326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:20.699445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:20.701182 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:20.701530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:20.701760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.702772 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.704400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.704991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.705650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:20.706461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.707605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.707605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:20.707605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.707605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:20.867094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:20.867345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:20.867345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:20.867550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:20.868215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:20.868389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.869036 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:20.892127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.893081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.895804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:20.896593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.897315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.897315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:20.897315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.897315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.898262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:20.899159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.900176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.900176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:20.900176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.900176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:20.970551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:20.971235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:20.971391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:20.971592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:20.972101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:20.972323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.972982 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:20.974542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.975577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.976168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.976804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.977176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:20.977176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.977176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:20.978086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:20.978550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:20.979071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:20.979071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:20.979071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:20.979071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:21.043729 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:21.044263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:21.044399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:21.044752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:21.045359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:21.045742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:21.045989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:21.046826 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:21.048197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:21.048920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:21.049624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:21.049783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:21.050620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:21.051607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:21.051607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:25.546402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:25.547112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:25.547243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:25.548143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:25.585553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:25.586400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.588467 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.589590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.590225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:25.590791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:25.591637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.592262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:25.592262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.592262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.592262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:25.592262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.592262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:25.699939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:25.700651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:25.700898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:25.702578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:25.702954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:25.703343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.704300 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.706449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.707149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:25.707653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.707653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.709131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.709606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:25.710247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.710247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:25.867787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:25.868301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:25.868437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:25.868583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:25.869537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:25.869865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.870788 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:25.872375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.873500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.874210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:25.875074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.876036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.876036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:25.876036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.876036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.876585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:25.877457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.878350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.878350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:25.878350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.878350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:25.971291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:25.971714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:25.971714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:25.971845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:25.972552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:25.972971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.973812 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:25.975226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.976047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.976593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.977259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.977958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:25.977958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.977958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:25.978321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:25.978883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:25.979428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:25.979428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:25.979428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:25.979428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:26.044240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:26.044834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:26.044899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:26.046500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:26.047082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:26.047557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:26.047812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:26.048374 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:26.049268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:26.049719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:26.050109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:26.050293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:26.050831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:26.051400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:26.051400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:30.547141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:30.547831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:30.547831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:30.548741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:30.584337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:30.584798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.586123 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.587793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.588166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:30.588521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.588521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:30.589531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.589882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.590329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:30.590329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.590329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:30.700808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:30.701344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:30.701439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:30.702496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:30.702964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:30.703273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.704184 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.705232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.705823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.706436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:30.707203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.708066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.708066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:30.708066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.708066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:30.868611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:30.869219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:30.869370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:30.869563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:30.870134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:30.870365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.870982 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:30.872393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.873094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.873935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:30.874517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.875094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.875094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:30.875094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.875094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.875739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:30.876227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.876731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.876731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:30.876731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.876731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:30.971787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:30.972409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:30.972519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:30.972646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:30.973360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:30.973835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.974691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:30.975814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.976615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.977312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.977868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.978458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:30.978458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.978458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:30.979044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:30.979706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:30.980595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:30.980595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:30.980595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:30.980595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:31.045165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:31.045940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:31.046111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:31.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:31.047143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:31.047558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:31.047813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:31.048706 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:31.050349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:31.050917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:31.051509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:31.051874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:31.052627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:31.053457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:31.053457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:35.548087 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:35.549177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:35.549341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:35.550663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:35.552762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=324,neutron:UPDATE=1 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:32:35.590813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:35.591276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.592403 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.593389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.593978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:35.594599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:35.595347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.596346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:35.596346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.596346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.596346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:35.596346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.596346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:35.701685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:35.702242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:35.702351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:35.703246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:35.703581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:35.703809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.704618 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.705668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.705945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:35.706216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.706216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.706666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.707041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:35.707383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.707383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:35.869595 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:35.870465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:35.870618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:35.870776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:35.871410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:35.871722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.872630 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:35.873934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.874695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.875211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:35.875752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.876376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.876376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:35.876376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.876376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.877844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:35.878703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.879638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.879638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:35.879638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.879638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:35.972536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:35.972979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:35.973176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:35.973370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:35.974352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:35.974816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.975821 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:35.977152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.977908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.978563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.979147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.979740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:35.979740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.979740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:35.980322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:35.981132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:35.981939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:35.981939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:35.981939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:35.981939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:36.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:36.046373 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:36.046518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:36.046791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:36.047446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:36.047861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:36.048134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:36.049028 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:36.050568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:36.051226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:36.051793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:36.052231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:36.053060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:36.053869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:36.053869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:40.548953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:40.549920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:40.549920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:40.551307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:40.590427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:40.590954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.593122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.593749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:40.594340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:40.595060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.595893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:40.595893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.595893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.595893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:40.595893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.595893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:40.702448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:40.702969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:40.703191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:40.704522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:40.704918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:40.705223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.706270 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.707928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.708560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:40.709184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.709184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.709546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.710375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:40.712600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.712600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:40.870717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:40.871574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:40.871725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:40.871886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:40.872825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:40.873199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.874111 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:40.875603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.876446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.876936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:40.877583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.878225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.878225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:40.878225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.878225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.878602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:40.879469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.880420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.880420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:40.880420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.880420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:40.973190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:40.973894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:40.974126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:40.974306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:40.975270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:40.975686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.976528 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:40.977971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.978767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.979449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.980023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:40.980636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:40.981566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:40.982404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:41.046343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:41.047128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:41.047282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:41.047563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:41.048512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:41.048937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:41.049176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:41.050099 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:41.051834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:41.052483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:41.053075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:41.053566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:41.054338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:41.055239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:41.055239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:45.549818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:45.553604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:45.553604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:45.553604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:45.588550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:45.588948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:45.591539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:45.591955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.592249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:45.592549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.593064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:45.703197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:45.703780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:45.703921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:45.705462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:45.705660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:45.706217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.706896 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.709050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.709418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:45.709766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.709766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.711561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.712141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:45.712660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.712660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:45.871671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:45.872225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:45.872354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:45.872516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:45.874479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:45.874479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.874753 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:45.876172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.877064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.879387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:45.880054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.880710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:45.881436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.882179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:45.974018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:45.974799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:45.974921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:45.975147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:45.975697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:45.975956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.976557 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:45.977739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.978405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.979163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.979625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.979970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:45.979970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.979970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:45.982041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:45.982542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:45.983068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:45.983068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:45.983068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:45.983068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:46.047103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:46.047764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:46.047913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:46.048217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:46.048884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:46.049636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:46.049636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:46.049916 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:46.051600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:46.052493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:46.053195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:46.054557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:46.055865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:46.055865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:46.055865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:50.551400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:50.552257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:50.552533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:50.553998 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:50.590630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:50.590849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.592037 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.594574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:50.595302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.595302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:50.596233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.597652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.598549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:50.598549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.598549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:50.703991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:50.704631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:50.704763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:50.706130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:50.706568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:50.706913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.707891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.709688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.710409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:50.711035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.711035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.711258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.712234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:50.713203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.713203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:50.872565 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:50.873054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:50.873208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:50.873433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:50.874155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:50.874504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.875426 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:50.876608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.877611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.878138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:50.878893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.879858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:50.880824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.882101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:50.974956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:50.975631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:50.975763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:50.975940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:50.976633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:50.977136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.978026 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:50.979350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.980428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.980925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.981873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:50.982744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:50.983814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:50.984974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:51.047991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:51.049132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:51.049132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:51.049132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:51.049772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:51.050188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:51.050523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:51.051256 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:51.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:51.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:51.054186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:51.054916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:51.055687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:51.056529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:51.056529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:55.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:55.552988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:55.553285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:55.554715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:55.586561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:55.586886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.587725 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.588599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.589311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:55.589899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:55.590766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.591761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:55.591761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.591761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.591761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:55.591761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.591761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:55.704812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:55.705292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:55.705568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:55.706682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:55.707083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:55.707347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.708221 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.709977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.710671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:55.711314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.711314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.711713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.713540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:55.714897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.714897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:55.873509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:55.873918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:55.873918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:55.874116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:32:55.874726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:55.875162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.876026 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:55.877224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.878089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.878575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:55.879218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.879889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:55.880663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.881643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:55.975826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:55.976486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:55.976798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:55.976798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:55.977617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:55.978668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:32:55.980579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.981395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.982229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.982838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.983477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:55.983477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.983477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:55.983848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:55.984627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:55.985418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:55.985418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:55.985418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:55.985418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:32:56.048719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:32:56.049127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:32:56.049317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:32:56.049594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:32:56.050206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:56.050638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:32:56.050890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:56.051885 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:56.053519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:32:56.054340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:56.054983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:32:56.055160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:32:56.056092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:32:56.057102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:32:56.057102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:00.552898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:00.553718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:00.553799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:00.555118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:00.591123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:00.591576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.592668 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.593724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.594298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:00.594987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:00.595689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:00.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:00.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:00.705720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:00.706480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:00.706480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:00.707640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:00.708001 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:00.708391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.709253 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.710405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.710756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:00.711099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.711099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.711765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.712237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:00.712726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.712726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:00.874132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:00.874798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:00.875027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:00.875261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:00.875982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:00.876432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.877424 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:00.879229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.879968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.880873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:00.881548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.882226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.882226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:00.882226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.882226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.882877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:00.883722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.884645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.884645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:00.884645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.884645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:00.976458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:00.977203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:00.977410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:00.977658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:00.978374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:00.978886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.979711 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:00.981200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.982091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.982440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.982952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:00.983610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:00.984346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:00.985249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:01.049385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:01.049993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:01.050172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:01.050448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:01.050941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:01.051356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:01.051582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:01.052348 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:01.053931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:01.054605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:01.055198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:01.055457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:01.056271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:01.057412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:01.057412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:05.553559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:05.554398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:05.554398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:05.555732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:05.598183 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:05.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.599615 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.600435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.600995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:05.601571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:05.602257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.603225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:05.603225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.603225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.603225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:05.603225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.603225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:05.706528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:05.706932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:05.707110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:05.708397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:05.708743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:05.708982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.709846 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.711556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.712229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:05.712778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.712778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.713617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.714278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:05.714774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.714774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:05.874977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:05.875260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:05.875416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:05.875571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:05.876514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:05.876895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.877751 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:05.878987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.879646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.880334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:05.880865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.881395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.881395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:05.881395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.881395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.882160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:05.882748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.883554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.883554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:05.883554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.883554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:05.977044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:05.977421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:05.977592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:05.977743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:05.978623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:05.979118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.980505 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:05.981791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.982666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.983658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.984529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.985239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:05.985239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.985239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:05.985398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:05.986291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:05.987473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:05.987473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:05.987473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:05.987473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:06.049975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:06.051459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:06.051459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:06.051459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:06.052198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:06.052615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:06.052856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:06.057045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:06.057860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:06.058528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:06.058528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:06.058528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:06.058528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:06.058528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:06.058915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:06.059748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:06.060661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:06.060661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:10.554302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:10.555090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:10.555175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:10.556502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:10.595989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:10.596179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.597105 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.597942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.598599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:10.599224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:10.599981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.600890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:10.600890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.600890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.600890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:10.600890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.600890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:10.707320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:10.707686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:10.707788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:10.709258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:10.709549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:10.709783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.710712 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.712277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.712905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:10.713529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.713529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.713987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.714858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:10.715720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.715720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:10.875430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:10.876021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:10.876221 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:10.876474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:10.877321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:10.877639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.878784 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:10.880178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.881027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.881472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:10.882128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.882727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.882727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:10.882727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.882727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.883230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:10.884065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.885034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.885034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:10.885034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.885034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:10.977720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:10.978297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:10.978448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:10.978620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:10.979355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:10.979767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.980810 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:10.982800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.983299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.984376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.984794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.985206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:10.985206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.985206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:10.986262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:10.986787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:10.987315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:10.987315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:10.987315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:10.987315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:11.050709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:11.051312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:11.051459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:11.051713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:11.052387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:11.052823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:11.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:11.053958 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:11.055579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:11.055977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:11.056405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:11.057475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:11.058218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:11.059415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:11.059415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:15.555049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:15.556357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:15.556569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:15.557907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:15.591816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:15.592345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.593327 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.594116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.594756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:15.595425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:15.596241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.597101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:15.597101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.597101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.597101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:15.597101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.597101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:15.708036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:15.708513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:15.708633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:15.709767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:15.710219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:15.710434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.711333 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.712647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.713269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.713803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:15.714517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.715376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.715376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:15.715376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.715376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:15.876192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:15.876671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:15.876831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:15.877118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:15.878044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:15.878514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.879539 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:15.881094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.881839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.882694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:15.883303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.883891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.883891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:15.883891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.883891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:15.885630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.886593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.886593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:15.886593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.886593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:15.978223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:15.978493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:15.978632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:15.978774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:15.979573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:15.980033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.980944 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:15.982179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.983189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.983719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:15.986222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:15.987289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:15.988648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:16.051452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:16.052317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:16.052557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:16.052939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:16.053610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:16.054032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:16.054277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:16.055061 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:16.056209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:16.056913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:16.057518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:16.058331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:16.059196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:16.059196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:16.059196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:20.555692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:20.556392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:20.556392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:20.557040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:20.593715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:20.594178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.595210 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.596332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.596956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:20.597599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:20.598340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.599358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:20.599358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.599358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.599358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:20.599358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.599358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:20.708946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:20.709407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:20.709545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:20.710802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:20.711181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:20.711367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.712198 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.713770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.714272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:20.714619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.714619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.715591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.716294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:20.716870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.716870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:20.877066 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:20.877654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:20.877803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:20.877960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:20.878621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:20.879059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.879917 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:20.881106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.881828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.882349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:20.883065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.883680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.883680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:20.883680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.883680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.883940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:20.884708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.885675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.885675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:20.885675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.885675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:20.978908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:20.979812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:20.980096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:20.980187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:20.981108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:20.981639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.982703 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:20.984253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.984987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.985675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.986229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.986829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:20.986829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.986829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:20.987577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:20.988259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:20.988777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:20.988777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:20.988777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:20.988777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:21.052419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:21.053088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:21.053242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:21.053523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:21.054241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:21.054699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:21.054954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:21.055959 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:21.057571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:21.058245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:21.059054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:21.059466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:21.060285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:21.061202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:21.061202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:25.556517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:25.557389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:25.557389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:25.558636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:25.597788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:25.599250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.599521 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.600666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:25.602611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:25.603444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.604746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:25.604746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.604746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.604746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:25.604746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.604746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:25.709877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:25.710283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:25.710421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:25.712874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:25.712874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:25.712874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.713158 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.714743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.715465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.716071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:25.716905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.717918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.717918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:25.717918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.717918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:25.877872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:25.878478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:25.878577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:25.878740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:25.880463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:25.880463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.880686 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:25.881984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.882568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.883104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:25.883507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.883865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.883865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:25.883865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.883865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.884606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:25.885096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.885656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.885656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:25.885656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.885656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:25.979804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:25.980398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:25.980530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:25.980710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:25.982419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:25.982419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.982647 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:25.983901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.985446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:25.986740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:25.987776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.988468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:25.989334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:26.053316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:26.053968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:26.054237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:26.055189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:26.057078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:26.057078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:26.057078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:26.057354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:26.058918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:26.059626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:26.060255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:26.061299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:26.062450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:26.063051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:26.063051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:30.557496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:30.558422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:30.558422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:30.559688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:30.594674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:30.595145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.596465 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.597519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.598143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:30.598711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:30.599525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.600673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:30.600673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.600673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.600673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:30.600673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.600673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:30.710684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:30.711294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:30.711462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:30.712635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:30.713047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:30.713391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.714536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.716040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:30.717250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.717250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.717713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.718516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:30.719352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.719352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:30.878800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:30.879340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:30.879461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:30.879582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:30.880466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:30.880850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.881731 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:30.883157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.883956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.884557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:30.885250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.886104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:30.886818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.887733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:30.980646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:30.980890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:30.980980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:30.981193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:30.981951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:30.982334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.983238 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:30.984768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.985498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.986209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.986757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.987391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:30.987391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.987391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:30.987887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:30.988724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:30.989213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:30.989213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:30.989213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:30.989213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:31.054084 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:31.054610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:31.054732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:31.055044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:31.055850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:31.056458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:31.056743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:31.057721 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:31.059275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:31.059951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:31.060603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:31.060774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:31.061615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:31.062473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:31.062473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:35.558452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:35.559620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:35.559620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:35.560683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:35.567858 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=241 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:33:35.597557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:35.598019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.599219 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.600523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.601212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:35.601872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:35.602672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.603602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:35.603602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.603602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.603602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:35.603602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.603602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:35.711468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:35.712088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:35.712260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:35.713258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:35.713544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:35.713713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.716114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.716725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:35.717344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.717344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.717795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.718593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:35.719465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.719465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:35.879575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:35.880209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:35.880366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:35.880531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:35.881307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:35.881657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.882585 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:35.884234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.884958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.885608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:35.886244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.886869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.886869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:35.886869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.886869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.887333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:35.888145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.889112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.889112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:35.889112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.889112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:35.981203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:35.981519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:35.981593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:35.981701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:35.982554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:35.983100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.983977 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:35.985264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.986075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.986601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.987207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.987768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:35.987768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.987768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:35.988170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:35.988900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:35.989732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:35.989732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:35.989732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:35.989732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:36.054808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:36.055512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:36.055661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:36.055958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:36.056641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:36.057506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:36.057960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:36.059210 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:36.060959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:36.061383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:36.061751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:36.062556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:36.063169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:36.063804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:36.063804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:40.559208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:40.560167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:40.560167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:40.561431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:40.602881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:40.603420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.604647 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.605985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.606624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:40.607295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:40.608078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.608970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:40.608970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.608970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.608970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:40.608970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.608970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:40.712209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:40.712800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:40.712929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:40.714378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:40.714763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:40.715025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.715969 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.717556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.718196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:40.718606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.718606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.719137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.719742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:40.720405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.720405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:40.880514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:40.881118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:40.881237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:40.881416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:40.882277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:40.882611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.883476 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:40.884884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.885759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.886207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:40.886610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.886989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.886989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:40.886989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.886989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.887834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:40.888407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.889287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.889287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:40.889287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.889287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:40.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:40.982678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:40.982852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:40.983047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:40.983715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:40.984129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.984936 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:40.986596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.987579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.988139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.988683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.989116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:40.989116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.989116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:40.989953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:40.990852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:40.991534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:40.991534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:40.991534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:40.991534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:41.055324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:41.055613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:41.055735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:41.055974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:41.056824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:41.057288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:41.057558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:41.058504 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:41.059984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:41.060673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:41.061280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:41.061601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:41.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:41.063221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:41.063221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:45.560025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:45.560911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:45.561207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:45.562564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:45.602509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:45.602953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.603936 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.605942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.606216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:45.606445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:45.606755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.607174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:45.607174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.607174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.607174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:45.607174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.607174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:45.712931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:45.713431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:45.713582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:45.714905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:45.715285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:45.715515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.716442 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.718057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.718610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:45.719136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.719136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.719720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.721265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:45.722911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.722911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:45.881266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:45.881592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:45.881592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:45.881928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:45.882769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:45.883227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.884680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:45.886332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.887145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.887908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:45.889353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.890609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:45.891292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.892753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:45.983084 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:45.983619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:45.983771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:45.983920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:45.984709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:45.985145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.986028 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:45.987339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.988048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.988730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.989960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:45.991073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:45.991709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:45.992474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:46.055817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:46.056299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:46.056528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:46.056748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:46.057509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:46.057905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:46.058148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:46.059166 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:46.060762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:46.061563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:46.062339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:46.063035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:46.063867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:46.064684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:46.064684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:50.560924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:50.561900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:50.561900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:50.563127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:50.602270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:50.602651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.603692 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.604602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.605111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:50.605817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:50.606318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.606851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:50.606851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.606851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.606851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:50.606851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.606851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:50.713722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:50.714419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:50.714612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:50.715875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:50.716349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:50.716608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.717507 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.718688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.719349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.720149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:50.721442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.722867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.722867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:50.722867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.722867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:50.881992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:50.882501 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:50.882613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:50.882705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:50.883648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:50.884021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.884859 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:50.885780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.886616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.886949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:50.887529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.888113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:50.888776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.889648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:50.983755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:50.984216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:50.984382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:50.984638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:50.985596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:50.986087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.987143 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:50.988571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.989348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.990315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.990887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.991483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:50.991483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.991483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:50.992139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:50.993042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:50.993832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:50.993832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:50.993832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:50.993832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:51.056339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:51.056929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:51.057089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:51.057340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:51.058210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:51.058627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:51.059014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:51.059844 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:51.061375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:51.061961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:51.062493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:51.062980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:51.063732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:51.064509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:51.064509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:55.561750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:55.562590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:55.562836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:55.564391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:55.602432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:55.602848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.603734 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.604651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.605071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:55.605437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:55.605882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.606438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:55.606438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.606438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.606438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:55.606438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.606438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:55.714481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:55.714910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:55.715132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:55.716121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:55.716449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:55.716634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.717826 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.719927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.720613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:55.721481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.721481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.721626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.722414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:55.723499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.723499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:55.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:55.898095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:55.898095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:55.898095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:33:55.898694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:55.898947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.899719 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:55.900412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.900792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.900997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:55.901407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.901699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.901699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:55.901699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.901699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.901798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:55.902225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.902781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.902781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:55.902781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.902781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:55.984319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:55.984870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:55.985034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:55.985265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:55.986185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:55.986680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.987572 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:33:55.989034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.989679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.990442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.991122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.991582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:55.991582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.991582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:55.992556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:55.993102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:55.993977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:55.993977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:55.993977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:55.993977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:33:56.056905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:33:56.057301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:33:56.057465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:33:56.057705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:33:56.058501 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:56.059271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:33:56.059490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:56.060872 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:56.062239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:33:56.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:56.063100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:33:56.063224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:33:56.063684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:33:56.064177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:33:56.064177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:00.562561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:00.576489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:00.576724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:00.578464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:00.619279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:00.619702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Traceback (most recent call last):
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.620580 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.621566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.622202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:00.622814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:00.623630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.624549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:00.624549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.624549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.624549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:00.624549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.624549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:00.715296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:00.715984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:00.716251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:00.717700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:00.718126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:00.718368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.719263 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.721023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.721480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:00.721960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.721960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.722986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.723640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:00.724333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.724333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:00.898137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:00.898856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:00.899186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:00.899322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:00.900432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:00.900806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.901708 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:00.902931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.903506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:00.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.905385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.905385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:00.905385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.905385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.906612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:00.907252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.908018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.908018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:00.908018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.908018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:00.985039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:00.985646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:00.985874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:00.986090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:00.986891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:00.987228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.988157 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:00.989498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.990332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.991427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.992042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.992679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:00.992679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.992679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:00.993158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:00.994393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:00.995403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:00.995403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:00.995403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:00.995403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:01.057424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:01.057879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:01.058030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:01.058317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:01.058931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:01.059462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:01.059769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:01.060648 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:01.062032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:01.062770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:01.063498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:01.063656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:01.064447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:01.065286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:01.065286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:05.576600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:05.577532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:05.577611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:05.579434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:05.615530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:05.615530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.616699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.617375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:05.617964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:05.618456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.619015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:05.619015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.619015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.619015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:05.619015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.619015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:05.716061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:05.716755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:05.717089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:05.719504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:05.719812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:05.720034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.721894 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.722448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:05.722966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.722966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.723620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.723938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:05.724289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.724289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:05.899176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:05.899764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:05.900129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:05.900129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.903680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.904492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:05.904993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:05.905420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.905828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:05.905828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.905828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.905828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:05.905828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.905828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.906821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:05.907493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.908148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.908148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:05.908148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.908148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:05.985935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:05.986619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:05.986775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:05.986944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:05.988058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:05.988979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.989809 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:05.991426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.992369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.992835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.993548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.994185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:05.994185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.994185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:05.995141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:05.995763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:05.996576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:05.996576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:05.996576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:05.996576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:06.059609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:06.059609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:06.059609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:06.059609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:06.060296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:06.060611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:06.060808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:06.061736 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:06.062852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:06.063263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:06.063581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:06.064491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:06.067285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:06.067850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:06.067850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:10.577299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:10.578282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:10.578282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:10.579181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:10.614959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:10.614959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.615210 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.615826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.616496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:10.617363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:10.618378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.620088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:10.620088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.620088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.620088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:10.620088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.620088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:10.717368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:10.717790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:10.717922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:10.719092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:10.720679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:10.720679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.720951 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.722561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.723236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:10.723843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.723843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.724258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.725270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:10.726057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.726057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:10.899931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:10.900683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:10.900839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:10.901027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:10.902731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:10.902731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.903082 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:10.904491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.905279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.905800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:10.906417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.907309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.907309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:10.907309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.907309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.907511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:10.908427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.909438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.909438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:10.909438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.909438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:10.986865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:10.987419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:10.987601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:10.987816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:10.990320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:10.990320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.990667 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:10.995035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.995593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.997515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.997978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.998402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:10.998402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.998402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:10.998618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:10.999151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:10.999697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:10.999697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:10.999697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:10.999697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:11.059478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:11.065690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:11.065840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:11.066133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:11.068114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:11.068114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:11.068114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:11.068364 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:11.069818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:11.070257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:11.070674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:11.071609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:11.072141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:11.072859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:11.072859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:15.578250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:15.579466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:15.579668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:15.580974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:15.619855 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:15.620350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.621332 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.622162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.622826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:15.623578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:15.624404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.625317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:15.625317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.625317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.625317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:15.625317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.625317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:15.717952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:15.718453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:15.718665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:15.719783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:15.720132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:15.720383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.721243 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.722699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.723208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:15.723617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.723617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.725049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.725540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:15.726112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.726112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:15.900728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:15.901395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:15.901535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:15.901724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:15.902442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:15.902697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.903428 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:15.904808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.905779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.906209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:15.906794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.907611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.907611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:15.907611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.907611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.908312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:15.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.909825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.909825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:15.909825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.909825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:15.987439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:15.987966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:15.988162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:15.988326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:15.989444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:15.989878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.990801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:15.992288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.993086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.993752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.994222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.994604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:15.994604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.994604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:15.996331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:15.996811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:15.997307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:15.997307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:15.997307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:15.997307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:16.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:16.066158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:16.066308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:16.066541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:16.067409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:16.067902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:16.068203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:16.069154 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:16.070673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:16.071792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:16.072353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:16.073244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:16.073915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:16.074655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:16.074655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:20.579151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:20.579976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:20.580276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:20.581989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:20.620573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:20.621027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.622055 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.622750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.623443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:20.624098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:20.624870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.625992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:20.625992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.625992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.625992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:20.625992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.625992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:20.718659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:20.719292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:20.719510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:20.720857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:20.721279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:20.721568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.722692 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.724797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.725461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:20.726274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.726274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.727267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.728140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:20.728940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.728940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:20.901661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:20.902103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:20.902325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:20.902481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:20.902912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:20.903196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.903773 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:20.904934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.905497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.906295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:20.906665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.907280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.907280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:20.907280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.907280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.907977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:20.908896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.909773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.909773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:20.909773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.909773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:20.988133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:20.988711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:20.988853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:20.989031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:20.989807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:20.990130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.990796 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:20.992208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.992903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.993634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.994352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.995137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:20.995137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.995137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:20.996065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:20.996950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:20.997728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:20.997728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:20.997728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:20.997728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:21.066841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:21.066936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:21.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:21.067331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:21.068164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:21.068846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:21.069142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:21.070107 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:21.072300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:21.072937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:21.073537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:21.075280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:21.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:21.077060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:21.077060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:25.580187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:25.583948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:25.583948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:25.583948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:25.616922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:25.617285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:25.620390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:25.620765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.621063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:25.621365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.621772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:25.719417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:25.720244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:25.720395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:25.721473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:25.721776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:25.722024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.722935 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.724531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.725229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:25.725789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.725789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.726528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.727392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:25.728259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.728259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:25.902288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:25.903454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:25.903454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:25.903454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:25.904102 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:25.904447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:25.908038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.909046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.909924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:25.910554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.911245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.911245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:25.911245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.911245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.912768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:25.913120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.913478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.913478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:25.913478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.913478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:25.988955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:25.989531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:25.989663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:25.989763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:25.990579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:25.990865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.991585 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:25.992527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.993420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:25.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:25.994661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.995074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:25.995819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:26.067030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:26.067350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:26.067441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:26.067623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:26.068751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:26.069241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:26.069494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:26.070460 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:26.071580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:26.072056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:26.072473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:26.073484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:26.074127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:26.075054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:26.075054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:30.581925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:30.582886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:30.583043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:30.584506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:30.628215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:30.628660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.629585 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.630505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.631081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:30.631642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:30.632388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:30.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:30.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:30.720253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:30.721013 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:30.721226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:30.722493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:30.722846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:30.723136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.724150 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.725811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.726724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:30.727383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.727383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.727759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:30.729661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.729661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:30.903281 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:30.903976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:30.904144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:30.904314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:30.905336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:30.905655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.906528 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:30.908287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.909105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.910571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:30.911326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.911745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.911745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:30.911745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.911745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:30.913393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:30.913948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.914559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.914559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:30.914559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.914559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:30.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:30.990934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:30.990934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:30.990934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:30.991978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:30.992503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.993689 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:30.997062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:30.997865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:31.000690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:31.001345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:31.001746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:31.002231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:31.067760 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:31.068695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:31.068901 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:31.069667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:31.070618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:31.071015 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:31.071243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.072086 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.073800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:31.074472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.074986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:31.077076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:31.077810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:31.078582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:31.078582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:35.582886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:35.583697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:35.583771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:35.585351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:35.587464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=239 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:34:35.626261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:35.626399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.627403 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.628806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.629647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:35.630347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:35.631204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.631977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:35.631977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.631977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.631977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:35.631977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.631977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:35.721109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:35.721692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:35.721846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:35.722914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:35.723284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:35.723519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.724373 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.725975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.726586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:35.727154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.727154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.727804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.728768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:35.729551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.729551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:35.904088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:35.904613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:35.904762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:35.904914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:35.905617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:35.905977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.906912 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:35.908271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.908987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.909543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:35.910110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.910650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.910650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:35.910650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.910650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.911270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:35.911847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.912445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.912445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:35.912445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.912445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:35.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:35.991143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:35.991313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:35.991654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:35.992337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:35.992787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.993688 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:35.995457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.995921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.996813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.997185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.997611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:35.997611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.997611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:35.998801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:35.999323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:35.999840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:35.999840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:35.999840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:35.999840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:36.068431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:36.068921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:36.069040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:36.069246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:36.070067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:36.070496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:36.070785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:36.071762 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:36.072703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:36.073366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:36.073972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:36.074330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:36.075143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:36.076715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:36.076715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:40.583653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:40.584877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:40.584877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:40.585646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:40.623069 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:40.623406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.624170 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.626236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:40.626981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.626981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:40.627388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.627848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.628399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:40.628399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.628399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:40.723245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:40.723245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:40.723245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:40.724802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:40.724802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:40.724802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.725075 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.727287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.727737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:40.729173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.729173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.729366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.729847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:40.730356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.730356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:40.904679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:40.905369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:40.905484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:40.905554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:40.906090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:40.906290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.906814 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:40.907540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.908473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:40.909255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.910101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:40.910779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.911811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:40.990931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:40.991347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:40.991434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:40.991624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:40.992309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:40.992768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.993671 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:40.994941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.995416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.996366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.997438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:40.997827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:40.998286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:40.998832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:41.069175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:41.069707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:41.069889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:41.070233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:41.070812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:41.071282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:41.071554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:41.072489 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:41.073891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:41.074633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:41.075293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:41.075930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:41.076858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:41.077747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:41.077747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:45.584320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:45.585469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:45.585546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:45.586868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:45.627264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:45.627264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.627680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.628561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.628956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:45.629331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.629331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:45.629617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.630581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.631268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:45.631268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.631268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:45.722823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:45.723076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:45.723186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:45.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:45.725150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:45.725150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.725310 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.726352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.726794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:45.727189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.727189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.727431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:45.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.728660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:45.905849 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:45.906623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:45.906778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:45.906913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:45.907872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:45.908536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.909382 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:45.910830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.911702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.912190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:45.912702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.913211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.913211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:45.913211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.913211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.913770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:45.914466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.915241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.915241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:45.915241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.915241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:45.991667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:45.992159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:45.992159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:45.992357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:45.993402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:45.993895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.994625 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:45.995838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.996438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.996650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.996983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.997423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:45.997423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.997423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:45.998193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:45.998827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:45.999378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:45.999378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:45.999378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:45.999378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:46.070101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:46.070406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:46.070543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:46.070785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:46.071352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:46.071813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:46.072078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:46.072857 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:46.074130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:46.074734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:46.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:46.075726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:46.076601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:46.077384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:46.077384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:50.585280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:50.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:50.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:50.586788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:50.625899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.627035 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.628131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.628584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:50.628860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.628860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:50.629288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.629899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.630358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:50.630358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.630358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:50.723314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:50.723913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:50.724964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:50.725189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.726326 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.727870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.728309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:50.728688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.728688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.729719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:50.730668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.730668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:50.906593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:50.907271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:50.907434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:50.907577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:50.908494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:50.908842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.909774 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:50.911098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.911974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.912424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:50.912990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.913643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:50.914345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.915269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:50.992151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:50.992390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:50.992462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:50.992584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:50.993261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:50.993828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.994724 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:50.995777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.996607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:50.997323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.997900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:50.998328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:50.999146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:51.000213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:51.000213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:51.000213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.000213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:51.070510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:51.071048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:51.071196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:51.071385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:51.071799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:51.072107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:51.072304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.072872 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:51.074341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:51.074954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.075531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:51.076101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:51.076812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:51.077618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:51.077618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:55.586202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:55.587091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:55.587237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:55.588344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:55.620071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:55.620254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.620801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.621512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.622186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:55.622773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:55.623543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.624454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:55.624454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.624454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.624454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:55.624454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.624454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:55.724164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:55.724835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:55.725172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:55.725918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:55.726317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:55.726583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.727534 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.729107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.729473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:55.729741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.729741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.731418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.731856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:55.732285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.732285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:55.907519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:55.908176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:55.908314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:55.908674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:34:55.909048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:55.909935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.911461 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:55.912797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.913299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.914206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:55.914490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.914760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.914760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:55.914760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.914760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.915031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:55.915581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.916202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.916202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:55.916202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.916202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:55.992943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:55.993856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:55.994346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:55.994346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:55.995182 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:55.995745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.996862 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:34:55.998295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.999416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:55.999882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:56.000810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:56.001703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:56.002367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.003150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:34:56.071686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:34:56.072119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:34:56.072219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:34:56.072438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:34:56.072864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:56.073218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:34:56.073358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.073987 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.075126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:34:56.076011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:34:56.076737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:34:56.077646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:34:56.078505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:34:56.078505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:34:56.078505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:00.586879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:00.587914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:00.587914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:00.589273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:00.628967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:00.629528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.630478 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.631442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.632307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:00.632737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:00.633230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.634353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:00.634353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.634353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.634353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:00.634353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.634353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:00.724900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:00.725701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:00.725701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:00.729159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:00.729415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:00.729573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.730205 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.731133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.731518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:00.731931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.731931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.732144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.732675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:00.733237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.733237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:00.908382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:00.908797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:00.908962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:00.909452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:00.909815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:00.910026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.910683 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:00.912516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.913127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.914019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:00.914592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.915124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.915124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:00.915124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.915124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:00.916266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:00.917030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.917696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.917696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:00.917696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.917696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:00.993829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:00.994429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:00.994588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:00.994761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:00.995541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:00.996259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.996889 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:00.998303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:00.999115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:01.001228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:01.001704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:01.002147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:01.002671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.003605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:01.072385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:01.072990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:01.073221 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:01.073465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:01.077548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:01.077925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:01.078183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.078781 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.081099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:01.081604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:01.082086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:01.082774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:01.083753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:01.083753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:01.083753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:05.587546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:05.588190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:05.588251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:05.589213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:05.632922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:05.633572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.634849 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.636490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.636877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:05.637305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.637305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:05.637600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.638117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.638678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:05.638678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.638678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:05.725629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:05.726874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:05.726874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:05.727630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:05.728196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:05.729097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.730221 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.731765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.732523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:05.732911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.732911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.734273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.734806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:05.735373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.735373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:05.909136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:05.909734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:05.909898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:05.910106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:05.910895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:05.911325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.912187 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:05.913644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.914271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.915123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:05.915559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.916087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.916087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:05.916087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.916087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.916833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:05.917495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.918145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.918145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:05.918145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.918145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:05.994429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:05.995158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:05.995237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:05.995321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:05.995965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:05.996426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.998202 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:05.998919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.999366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.999610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:05.999928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:06.000349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:06.000349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.000349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.000447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:06.000929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:06.001452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:06.001452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:06.001452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.001452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:06.073409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:06.073687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:06.073687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:06.073872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:06.074314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:06.074610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:06.074850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.076227 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:06.078133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:06.078957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:06.079789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:06.080610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:06.081610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:06.081610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:06.081610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:10.588389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:10.589217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:10.589414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:10.591775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:10.650961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:10.651431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.652480 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.653670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.654342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:10.654873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.654873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:10.655722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.656568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.657542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:10.657542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.657542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:10.726538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:10.727048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:10.727348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:10.728187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:10.728359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:10.728633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.729344 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.730457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.731220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:10.731620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.731620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.732187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.732688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:10.733191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.733191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:10.910163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:10.910878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:10.910878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:10.910878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:10.912140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:10.912785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:10.915477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.915964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.917595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:10.917989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.918406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.918406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:10.918406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.918406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:10.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:10.918988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.919525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.919525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:10.919525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.919525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:10.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:10.995980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:10.996119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:10.996317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:10.997058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:10.997370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:10.998204 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:11.000286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.000992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:11.002402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:11.003064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:11.003470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:11.003470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.003470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.004805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:11.005392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:11.005962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:11.005962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:11.005962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.005962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:11.074166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:11.074942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:11.075133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:11.075409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:11.076193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:11.076516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:11.076692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.077497 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.078788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:11.079282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.079653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:11.080058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:11.080600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:11.081219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:11.081219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:15.594032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:15.641044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:15.641044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:15.642667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:15.682491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:15.682491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.683045 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.683865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.684427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:15.685203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:15.685580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.686135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:15.686135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.686135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.686135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:15.686135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.686135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:15.731820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:15.731820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:15.731820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:15.733351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.735127 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.735777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.736063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:15.736357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.736357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.737715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.738371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:15.739189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.739189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:15.911362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:15.912499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:15.912644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:15.912817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:15.914305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:15.914702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.915691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:15.917198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.918045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.918433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:15.919074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.919733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.919733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:15.919733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.919733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:15.920028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:15.920926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.921556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.921556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:15.921556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.921556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:15.996148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:15.996761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:15.996894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:15.997414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:15.998231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:15.998729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:15.999680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:16.000945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.001776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:16.002182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:16.002781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:16.003165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:16.003165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.003165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.003789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:16.004378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:16.005411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:16.005411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:16.005411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.005411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:16.075344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:16.075852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:16.075852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:16.076239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:16.076921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:16.077449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:16.077708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.080492 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.082029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:16.082658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.083491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:16.083929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:16.084624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:16.085310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:16.085310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:20.637860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:20.639055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:20.639055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:20.693902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:20.729207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:20.729685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.730643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:20.730883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:20.730951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.732170 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_fdb_aging_settings failed after 0.094 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.733921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.734224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:20.734504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.734504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:20.734554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:20.735997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.736366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.736733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:20.736733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.736733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:20.736733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:20.736733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.736856 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.737546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.737900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:20.738170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.738170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.738326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.738845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:20.739399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.739399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:20.912497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:20.913034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:20.913124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:20.913251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:20.913825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:20.914102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.915029 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:20.916596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.917483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.918030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:20.918654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.919305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.919305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:20.919305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.919305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:20.919963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:20.920617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:20.921223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:20.921223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:20.921223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:20.921223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:20.996929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:20.997991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:20.998360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:20.998475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:20.999143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:20.999544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.000288 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:21.001218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.001939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:21.002317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.002873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.003523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:21.003523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.003523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.005299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:21.005790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.006356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.006356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:21.006356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.006356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:21.076887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:21.077618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:21.077791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:21.078140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:21.078893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:21.079415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:21.079661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.080862 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.082703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:21.083729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.084337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:21.085223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:21.085716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:21.086213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:21.086213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:25.638716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:25.639865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:25.639865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:25.641302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:25.683353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:25.683619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.684238 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.685411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:25.685801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:25.686435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.686995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:25.686995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.686995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.686995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:25.686995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.686995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:25.731640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:25.732446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:25.732537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:25.733778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:25.734129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:25.734377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.735501 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.736486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.737155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:25.737764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:25.738536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.739190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:25.739190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.739190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.739190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:25.739190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.739190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:25.913406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:25.914396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:25.914396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:25.914863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:25.916337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:25.916337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.917375 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:25.919025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.919865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.920533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:25.921161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.921764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.921764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:25.921764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.921764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:25.922275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:25.923269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:25.924203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:25.924203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:25.924203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:25.924203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:25.997880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:25.998401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:25.998556 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:25.998790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:25.999597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:25.999939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.000612 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:26.001400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.002203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:26.002750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.003304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:26.003893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:26.004628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.005424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:26.077547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:26.078287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:26.078359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:26.078590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:26.079240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:26.079642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:26.079884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.080641 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.081861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:26.082556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:26.083165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:26.083904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:26.084732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:26.084732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:26.084732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:30.639513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:30.640441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:30.640441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:30.641794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:30.682594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:30.683114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.684195 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.685029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.685738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:30.686450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:30.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.688336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:30.688336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.688336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.688336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:30.688336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.688336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:30.732498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:30.733127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:30.733262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:30.734445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:30.734874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:30.735231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.736294 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.737571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.738067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:30.738413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.738413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:30.739230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.739688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.740230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:30.740230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.740230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:30.914432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:30.915075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:30.915168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:30.915276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:30.915761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:30.916082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.916712 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:30.917610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.918630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:30.919438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.920157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:30.920844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:30.922104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:30.998473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:30.998812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:30.998944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:30.999153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:30.999840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:31.000199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.001446 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:31.002986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.003607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:31.004511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.005025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.005539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:31.005539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.005539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.006227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:31.006761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.007351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.007351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:31.007351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.007351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:31.078500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:31.078912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:31.078969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:31.079180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:31.080132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:31.080461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:31.080657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.081397 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.082377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:31.082812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:31.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:31.083969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:31.084512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:31.084512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:31.084512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:35.640433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:35.641388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:35.641669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:35.643161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:35.645438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:35:35.678165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:35.678707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.679726 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.680876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.681700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:35.682496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:35.683500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.684595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:35.684595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.684595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.684595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:35.684595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.684595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:35.733458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:35.733984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:35.734165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:35.735081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:35.735267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:35.735444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.736144 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.737450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.738055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:35.738600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.738600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:35.738967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.739632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.740460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:35.740460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.740460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:35.915100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:35.916918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:35.917067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:35.917230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:35.918128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:35.918519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.919400 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:35.920342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.920865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.921389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:35.921776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.922174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.922174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:35.922174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.922174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:35.922539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:35.923098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:35.923602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:35.923602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:35.923602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:35.923602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:35.999307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:36.000098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:36.000205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:36.000419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:36.001066 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:36.001396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.002263 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:36.003174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.003506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:36.004081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.004373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.004705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:36.004705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.004705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.005164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:36.005495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.005848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.005848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:36.005848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.005848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:36.079328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:36.079833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:36.079941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:36.080167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:36.081104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:36.081354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:36.081354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.081867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.082862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:36.083488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:36.084090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:36.084958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:36.085580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:36.085580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:36.085580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:40.641298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:40.642316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:40.642316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:40.643685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:40.684480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:40.684944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.685935 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.687172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.687470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:40.687741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.687741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:40.688308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.688656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.689059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:40.689059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.689059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:40.734382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:40.735070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:40.735211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:40.736370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:40.736890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:40.737170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.738078 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.739388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.739738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:40.740073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.740073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:40.740784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.741232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.741670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:40.741670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.741670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:40.916898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:40.917359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:40.917421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:40.917482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:40.918093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:40.918368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.919487 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:40.920141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.920831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:40.921492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.922231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:40.922806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:40.923673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:41.000294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:41.000874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:41.000998 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:41.001209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:41.001862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:41.002342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.003673 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:41.005018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.005706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:41.006474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.007082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.007643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:41.007643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.007643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.007992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:41.009094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.010182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.010182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:41.010182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.010182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:41.079913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:41.080571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:41.080681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:41.080951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:41.081572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:41.081953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:41.082229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.083152 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.084577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:41.085471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:41.086221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:41.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:41.087976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:41.087976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:41.087976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:45.642039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:45.643078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:45.643078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:45.644557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:45.687443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:45.687727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.688587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.689869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.690279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:45.690629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.690629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:45.690938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.691442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.692346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:45.692346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.692346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:45.735285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:45.735690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:45.735867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:45.736897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:45.737185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:45.737375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.738189 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.739870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.740265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:45.740622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.740622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:45.741697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.742148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.742668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:45.742668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.742668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:45.917726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:45.918231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:45.918367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:45.918528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.973095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.973706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:45.974194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:45.974574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.975035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:45.975035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.975035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.975035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:45.975035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.975035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:45.975278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:45.975794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:45.976342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:45.976342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:45.976342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:45.976342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:46.000989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:46.001626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:46.001784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:46.001969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:46.002706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:46.003198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.003998 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:46.005267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.006106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:46.006655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.007915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:46.007915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.007915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.008546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:46.009389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.010323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.010323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:46.010323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.010323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:46.080573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:46.081317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:46.081603 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:46.081969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:46.082661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:46.083147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.084278 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.086268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:46.086922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.087682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:46.088270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:46.088795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:46.089327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:46.089327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:50.642744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:50.643557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:50.643680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:50.645113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:50.679572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:50.680022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.680992 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.682074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.682848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:50.684258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:50.685056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.685940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:50.685940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.685940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.685940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:50.685940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.685940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:50.735878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:50.736265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:50.736364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:50.737615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:50.738164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:50.738358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.739305 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.740443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.741072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:50.741659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:50.742345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.743200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:50.743200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.743200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.743200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:50.743200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.743200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:50.918357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:50.918987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:50.919167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:50.919322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:50.920134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:50.920394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.921103 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:50.922426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.922947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.923708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:50.924105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.924573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.924573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:50.924573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.924573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:50.925075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:50.925604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:50.926152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:50.926152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:50.926152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:50.926152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:51.039395 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:51.039966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:51.040863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:51.041454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.041962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.042430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:51.043833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.044539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.044539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:51.044539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.044539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:51.081345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:51.081957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:51.082152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:51.082381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:51.082997 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:51.083430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:51.083993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.084705 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.086280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:51.086972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.087654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:51.088976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:51.089817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:51.090679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:51.090679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:55.643622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:55.644457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:55.644457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:55.645802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:55.682879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:55.683225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.683979 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.685300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.685921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:55.686584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:55.687238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.688054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:55.688054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.688054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.688054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:55.688054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.688054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:55.737736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:55.737736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:55.737736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:55.738407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:55.738850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:55.739147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:55.743046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:55.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.744528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:55.744528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.744528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.744528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:55.744528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.744528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:55.745514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.746301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.747139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:55.747139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.747139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:55.919226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:55.919631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:55.920475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:55.921077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:35:55.922108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:55.922479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.923487 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:55.925148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.925468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.926875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:55.927184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.927475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.927475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:55.927475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.927475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:55.928099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:55.928423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:55.928830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:55.928830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:55.928830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:55.928830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:56.037936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:56.038747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:56.039153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:56.039343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:56.098847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:56.098847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.099435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:35:56.099948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:35:56.100121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.100835 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:35:56.102155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.102574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:56.103106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.103623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:56.103623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.103623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:56.103623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.105063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:56.105759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.106297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.106297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:56.106297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.106297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:35:56.106297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:35:56.106297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.106833 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.107935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:35:56.108343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.108706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:35:56.109040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:35:56.109719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:35:56.110264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:35:56.110264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:00.644383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:00.648601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:00.648601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:00.648601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:00.684761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:00.685259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.686184 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.688205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.688524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:00.688814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:00.689272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.689694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:00.689694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.689694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.689694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:00.689694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.689694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:00.737264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:00.737972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:00.738131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:00.739197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:00.739502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:00.739663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.740369 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.741566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.742252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:00.742883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:00.743654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.744632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:00.744632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.744632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.744632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:00.744632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.744632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:00.919830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:00.920712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:00.920817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:00.921092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:00.922168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:00.922657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.923766 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:00.925424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.926354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.926868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:00.927510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.928346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:00.929769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:00.931116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:01.038929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:01.039513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:01.039665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:01.040094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:01.040945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:01.041445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.042155 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:01.043526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.044474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:01.046059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.046427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.046774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:01.046774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.046774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.046914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:01.047414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.047917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.047917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:01.047917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.047917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:01.100788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:01.100788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:01.100788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:01.100788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:01.101535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:01.101820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:01.102135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.103375 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.104674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:01.105355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.105839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:01.107292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:01.107869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:01.108425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:01.108425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:05.646224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:05.650238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:05.650238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:05.650238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:05.685596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:05.685888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:05.688781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:05.689436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.689951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:05.690465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.691210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:05.738174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:05.738982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:05.739178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:05.740062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:05.740472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:05.740805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.741807 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.742757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.743615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:05.744359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:05.744858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.745538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:05.745538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.745538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.745538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:05.745538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.745538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:05.921139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:05.921748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:05.921995 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:05.922101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:05.922819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:05.923139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.923933 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:05.925499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.926224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.926770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:05.927372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.928091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.928091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:05.928091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.928091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:05.929106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:05.929590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:05.930221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:05.930221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:05.930221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:05.930221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:06.039895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:06.040477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:06.040573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:06.040697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:06.041366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:06.041679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.042331 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:06.043085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.043806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:06.044525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:06.045188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:06.045734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.046955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:06.101474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:06.102267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:06.102868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:06.102868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:06.103691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:06.104120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:06.104263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.104912 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.105898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:06.106481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.106847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:06.107164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:06.107747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:06.108399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:06.108399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:10.648103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:10.648757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:10.648854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:10.649824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:10.695706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:10.696057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.696981 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.697867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.698208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:10.698583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.698583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:10.698743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.699179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.699817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:10.699817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.699817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:10.738811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:10.739131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:10.739434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:10.740670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:10.740895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:10.741036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.741650 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.742799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.743144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:10.743397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.743397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:10.743830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.744214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.744551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:10.744551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.744551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:10.921861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:10.923372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:10.923493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:10.923647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:10.924388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:10.924675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.926445 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:10.927761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.928277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.930328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:10.930849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.931258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.931258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:10.931258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.931258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:10.931822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:10.932571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:10.933150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:10.933150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:10.933150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:10.933150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:11.040771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:11.042109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:11.042109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:11.042109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:11.043158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:11.043545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.044354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:11.045101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:11.045764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.046157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.046556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:11.046556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.046556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.047411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:11.048053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.048625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.048625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:11.048625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.048625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:11.102325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:11.102796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:11.102888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:11.103084 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:11.103651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:11.104061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:11.104378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.105113 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.106112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:11.106546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.106921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:11.107238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:11.107740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:11.108266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:11.108266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:15.648875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:15.649833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:15.649833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:15.650097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:15.694823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:15.695287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.696024 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.697156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.697758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:15.698810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:15.699741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.700524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:15.700524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.700524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.700524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:15.700524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.700524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:15.739484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:15.740163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:15.740249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:15.741126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:15.741343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:15.741526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.742312 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.743021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.743726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:15.744260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:15.744994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.746196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:15.746196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.746196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.746196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:15.746196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.746196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:15.923577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:15.924098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:15.924178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:15.924304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:15.924891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:15.925477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.926097 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:15.926771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.927276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.927523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:15.927839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.928202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.928202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:15.928202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.928202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:15.928527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:15.928993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:15.929623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:15.929623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:15.929623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:15.929623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:16.041495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:16.042078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:16.042151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:16.042280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:16.043121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:16.043545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.044430 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:16.045373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.046334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:16.047161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.047857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.048342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:16.049140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.050166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.050166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:16.050166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.050166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:16.102952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:16.103368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:16.103467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:16.103670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:16.104333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:16.104627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:16.104784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.105503 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.106965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:16.107830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:16.108490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:16.109282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:16.110143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:16.110143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:16.110143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:20.649383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:20.650138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:20.650370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:20.651579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:20.695276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:20.695847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.696904 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.697723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.698828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:20.699219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:20.699594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.700310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:20.700310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.700310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.700310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:20.700310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.700310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:20.740324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:20.741129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:20.741327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:20.742524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:20.742807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:20.743484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.744012 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.746104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.746783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:20.747169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.747169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:20.748694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.749321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.750044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:20.750044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.750044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:20.924607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:20.924937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:20.925067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:20.925145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:20.925806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:20.926080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.926719 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:20.927775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.928503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:20.928955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.929421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:20.929792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:20.930348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:20.930903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:20.930903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:20.930903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:20.930903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:21.042408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:21.042813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:21.043088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:21.043147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:21.044020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:21.044536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.045433 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:21.046443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.046853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:21.048433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.048773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.049121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:21.049121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.049121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.049764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:21.050216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.050653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.050653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:21.050653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.050653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:21.103352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:21.104026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:21.104212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:21.104451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:21.105191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:21.105661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:21.105924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.106989 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.108240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:21.108868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:21.109471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:21.110230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:21.111104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:21.111104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:21.111104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:25.650155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:25.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:25.650745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:25.651785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:25.685832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:25.686159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.686893 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.687769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.688204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:25.688566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:25.688962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.689747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:25.689747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.689747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.689747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:25.689747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.689747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:25.741205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:25.741876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:25.742248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:25.743271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:25.743595 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:25.743779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.744471 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.745429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.745689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:25.745930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.745930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:25.746279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.746604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.746953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:25.746953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.746953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:25.925249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:25.925717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:25.925889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:25.925889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:25.926608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:25.926797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.927340 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:25.929352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.930554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:25.931345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.932032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:25.932679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:25.933714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:26.043180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:26.044417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:26.044640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:26.044640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:26.045251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:26.045468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.046034 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:26.046749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.047293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:26.047530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.047962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:26.049513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:26.050139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.050723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:26.104054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:26.105128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:26.105253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:26.105392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:26.106097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:26.106433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:26.106601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.107326 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.108327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:26.108793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:26.109209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:26.109737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:26.110525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:26.110525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:26.110525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:30.650778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:30.651464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:30.651464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:30.652852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:30.688041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:30.688513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.689605 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.690458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.691114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:30.691730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:30.692481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.693473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:30.693473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.693473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.693473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:30.693473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.693473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:30.742068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:30.742990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:30.743265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:30.744592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:30.745030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:30.745313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.746346 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.748120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.748447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:30.748723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.748723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:30.749815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.750500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.750877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:30.750877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.750877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:30.926071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:30.926228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:30.926294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:30.926384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:30.927308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:30.927582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.928320 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:30.932040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.932492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:30.932865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.933174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:30.933663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:30.934094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:31.044492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:31.045151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:31.045225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:31.045404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:31.046142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:31.046438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.047195 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:31.050384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.050939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:31.051430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:31.051863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:31.052316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.052933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:31.105351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:31.106157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:31.106157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:31.106157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:31.106929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:31.107176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:31.107344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.107967 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.108765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:31.109205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:31.109672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:31.110147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:31.110725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:31.110725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:31.110725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:35.651457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:35.652321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:35.652321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:35.653151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:35.654799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:36:35.689106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:35.689403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.690125 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.691502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.692030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:35.692404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:35.692826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.693401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:35.693401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.693401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.693401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:35.693401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.693401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:35.742881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:35.743635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:35.743887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:35.744930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:35.745329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:35.745579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Traceback (most recent call last):
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.746470 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.747988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.748609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:35.749169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:35.749865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.750678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:35.750678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.750678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.750678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:35.750678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.750678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:35.926572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:35.927480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:35.927622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:35.927729 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:35.928522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:35.928735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.929349 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:35.930423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.930870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.931389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:35.931734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.932221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.932221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:35.932221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.932221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:35.932606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:35.933072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:35.933559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:35.933559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:35.933559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:35.933559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:36.045284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:36.046128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:36.046665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:36.046774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:36.048416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:36.048732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.049536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:36.050340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.051083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:36.051518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.051955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.052353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:36.052353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.052353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.052448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:36.052899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.053899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.053899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:36.053899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.053899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:36.106255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:36.106560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:36.106644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:36.106820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:36.107692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:36.108190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:36.108440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.109396 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.110407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:36.111093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:36.111744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:36.112533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:36.113345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:36.113345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:36.113345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:40.652073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:40.652717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:40.652913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:40.654277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:40.688093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:40.688632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.689547 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.690416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.691111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:40.691719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:40.692423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.693413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:40.693413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.693413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.693413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:40.693413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.693413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:40.743629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:40.744043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:40.744226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:40.745344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:40.745750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:40.746218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Traceback (most recent call last):
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.747316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.748555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.748843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:40.749144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.749144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:40.750179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.750475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.750818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:40.750818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.750818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:40.927484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:40.927864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:40.927952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:40.928096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:40.928569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:40.928796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.929448 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:40.930419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.931091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.931318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:40.931646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.932021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.932021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:40.932021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.932021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:40.932557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:40.933079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:40.933554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:40.933554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:40.933554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:40.933554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:41.046333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:41.046768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:41.046947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:41.047100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:41.047712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:41.048463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.049431 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:41.050783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.051832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:41.052491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.053380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:41.054355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:41.055394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.056591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:41.106990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:41.107634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:41.107717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:41.108220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:41.108515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:41.109367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:41.109367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.110075 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.110960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:41.111347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:41.111680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:41.113057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:41.113536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:41.113536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:41.113536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:45.492751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:45.493470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:45.493718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:45.503275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 02:36:45.503601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 02:36:45.503887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.534039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.534453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.534842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.534977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.535759 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.536764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.537099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.537401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.537401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.537401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.537401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.567337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, resource={'id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['97900556-2eeb-4240-89bd-ed8258b9022d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:48Z', 'updated_at': '2026-04-13T02:30:49Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9202}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8120b9cd-5d7c-4d4d-9a57-22a6c716f312, external_ids_update=None, columns={'tag': 9202}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.570454 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Failed to fix resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.570821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.571122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.571405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.571405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.571405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.571405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.603272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.603672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.604521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.604684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.606133 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.607520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.607975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.608596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.608596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.608596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.608596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.648816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.649097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.650310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.650521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.651424 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.652894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.653523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.654100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.654100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.654100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.654100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource a4b11309-cd09-4af2-9981-d7a685003279 (type: security_groups) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.654100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:45.654905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:45.654905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:45.655912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:45.718050 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.719285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.720196 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_fdb_aging_settings failed after 0.066 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.723547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.723994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.724675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.724675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:45.725601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.726212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.726899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:45.726899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.726899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:45.729895 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.db.ovn_revision_numbers_db [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Successfully bumped revision number for resource a4b11309-cd09-4af2-9981-d7a685003279 (type: security_groups) to 10
Apr 13 02:36:45.730180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource 10dc7768-37ad-4931-8e70-b58cef0ccbc6 (type: security_groups) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.745598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:45.746261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:45.746487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:45.748201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.748655 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.748987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Traceback (most recent call last):
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.750891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.754022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.754444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.754825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.754825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:45.757489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.758018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.758764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:45.758764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.758764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:45.792099 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.db.ovn_revision_numbers_db [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Successfully bumped revision number for resource 10dc7768-37ad-4931-8e70-b58cef0ccbc6 (type: security_groups) to 15
Apr 13 02:36:45.792277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.905091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, resource={'id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'enable_snat': True}, 'gw_port_id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-exgz1', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'ip_address': '172.24.5.236', 'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, {'ip_address': '2001:db8::2c9', 'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 3, 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46'}, resource_type=routers, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.905433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.905657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.905810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.905939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Unable to update router 530b037d-a47e-4f63-bd55-041139eeee2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Failed to fix resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.908920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.909481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.909981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.910544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.910544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.910544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.910544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.910544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Fixing resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:36:45.938239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:45.940862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:45.941310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:45.941310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.948044 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:45.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.949924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.951812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:45.952293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.952801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.952801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:45.952801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.952801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:45.954555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:45.954968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:45.955446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:45.955446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:45.955446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:45.955446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:46.049789 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:46.050762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:46.054942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:46.055477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:46.057156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:46.057565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.058899 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:46.060787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.061541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:46.063881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.064643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.065484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:46.065484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.065484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.066200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:46.067274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.068301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.068301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:46.068301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.068301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:46.077200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, resource={'id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'name': '', 'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'project_id': '', 'mac_address': 'fa:16:3e:92:a4:24', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000165409', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:46.077493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.236/24', '2001:db8::2c9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:46.077779 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3ec69888-b9c9-4830-9d48-853dd81cf68b, lrouter_port=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:46.077985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.083405 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Maintenance task: Failed to fix resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:46.085234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.085597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.086019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:46.086019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.086019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:46.086019 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task check_for_inconsistencies finished in 0.591 seconds
Apr 13 02:36:46.107731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:46.108152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:46.108207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:46.108405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:46.108948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:46.109286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:46.109577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.110254 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.111084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:46.111551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.111912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:46.112129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:46.112656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:46.113944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:46.113944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:50.655256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:50.656037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:50.656253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:50.659212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:50.697498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:50.697695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.698481 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.699605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.699895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:50.700172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.700172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:50.700694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.701039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.701379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:50.701379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.701379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:50.746424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:50.747250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:50.747382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:50.748317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:50.748814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:50.748814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.749275 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.750327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.751041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:50.751520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:50.752403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.752938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:50.752938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.752938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.752938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:50.752938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.752938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:50.940763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:50.941255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:50.941336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:50.941465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:50.942134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:50.942499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.943606 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:50.944382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.945240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.945985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:50.946590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.947337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.947337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:50.947337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.947337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:50.947873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:50.948559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:50.949120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:50.949120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:50.949120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:50.949120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:51.050900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:51.051619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:51.051709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:51.052135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:51.052797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:51.052797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.053224 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:51.054100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.055092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:51.056121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:51.056969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:51.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.058498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:51.108481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:51.109214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:51.109316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:51.109518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:51.110207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:51.110635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:51.110801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.111705 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.112612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:51.113309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:51.113939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:51.114741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:51.115703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:51.115703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:51.115703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:55.656047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:55.656793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:55.657891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:55.658417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:55.697348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:55.697677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.698271 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.699247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.699822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:55.700314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:55.700797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.701416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:55.701416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.701416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.701416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:55.701416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.701416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:55.747706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:55.748103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:55.748614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:55.749368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:55.749648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:55.749853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.750693 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.751848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.752306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:55.752637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.752637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:55.753393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.753929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.754516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:55.754516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.754516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:55.941610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:55.942465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:55.942465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:55.942465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:36:55.943072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:55.943471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.944334 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:55.945312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.946551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:55.947384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.948127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:55.948744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:55.949698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:56.051824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:56.052671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:56.052671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:56.052847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:56.053663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:56.054187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.055014 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:36:56.055968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.056772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:56.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.057829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:56.059294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:56.059994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.061223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:36:56.109460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:36:56.110112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:36:56.110365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:36:56.110620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:36:56.111332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:56.111826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:36:56.112201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.113061 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.114053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:36:56.114610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:36:56.114980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:36:56.115508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:36:56.116055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:36:56.116055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:36:56.116055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:00.656821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:00.657463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:00.657463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:00.657796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:00.686932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:00.687252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.687983 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.689126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.689835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:00.690131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.690131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:00.690652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.691095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.691572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:00.691572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.691572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:00.748581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:00.749303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:00.749427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:00.750869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:00.751315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:00.751685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.752574 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.753499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.754137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:00.755360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:00.755981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.756748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:00.756748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.756748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.756748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:00.756748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.756748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:00.942567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:00.942916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:00.943026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:00.943156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:00.943838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:00.944107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.944763 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:00.945479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.945813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.946246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:00.946510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.946761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.946761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:00.946761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.946761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:00.947235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:00.947630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:00.948052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:00.948052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:00.948052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:00.948052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:01.052957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:01.053453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:01.053578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:01.053788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:01.054837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:01.055432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.056617 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:01.057961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.058797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:01.060029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.060927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.061696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:01.061696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.061696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.062068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:01.062808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.063364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.063364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:01.063364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.063364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:01.110437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:01.110766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:01.110846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:01.110992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:01.111765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:01.112199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:01.112474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.113272 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.114043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:01.114478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:01.115391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:01.116416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:01.117250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:01.117250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:01.117250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:05.657386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:05.658161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:05.658391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:05.659320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:05.698938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:05.699128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.699897 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.701145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.701662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:05.702128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:05.702755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.703506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:05.703506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.703506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.703506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:05.703506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.703506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:05.750333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:05.750333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:05.750607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:05.752130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:05.752669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:05.753032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.754114 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.755275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.755872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:05.756600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.756600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:05.756705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.757545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.758432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:05.758432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.758432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:05.943289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:05.943667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:05.943823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:05.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:05.944598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:05.944913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.945727 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:05.947106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.947912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.948492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:05.949142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.949831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.949831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:05.949831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.949831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:05.950240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:05.951310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:05.952305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:05.952305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:05.952305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:05.952305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:06.053761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:06.054529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:06.054703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:06.054981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:06.055815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:06.056291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.057485 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:06.058965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.059613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:06.060161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.060724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:06.061392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:06.062182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.062766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:06.111200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:06.111719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:06.111812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:06.112023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:06.113111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:06.113386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:06.113641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.114718 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.116182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:06.116675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.117072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:06.117546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:06.118107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:06.118723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:06.118723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:10.658130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:10.658782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:10.658782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:10.659478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:10.696137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:10.697454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.698334 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.699672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.700414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:10.700754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.700754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:10.701521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.702044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.702550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:10.702550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.702550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:10.751175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:10.752339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:10.752339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:10.752778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:10.753226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:10.753582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.754988 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.755898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.756319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:10.756653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.756653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:10.757498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.757855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.758229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:10.758229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.758229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:10.943794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:10.944397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:10.944397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:10.944570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:10.945782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:10.946029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.947616 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:10.949358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.949986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:10.950302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.950621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:10.950876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:10.952626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:11.054841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:11.055728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:11.055927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:11.056111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:11.057039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:11.057178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.057954 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:11.060313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.061143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:11.064097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.064646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.065215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:11.065215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.065215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.066144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:11.066849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.067558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.067558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:11.067558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.067558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:11.111887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:11.112180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:11.112256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:11.112442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:11.113207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.114591 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.115768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:11.116400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:11.116953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:11.117707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:11.118254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:11.118254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:11.118254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:15.670816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:15.671245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:15.671359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:15.672214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:15.707795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:15.708020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.708749 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.709399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.710163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:15.710538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:15.711061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.711640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:15.711640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.711640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.711640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:15.711640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.711640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:15.751631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:15.752367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:15.752553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:15.753663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:15.754053 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:15.754319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.755278 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.756580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.756947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:15.757330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.757330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:15.758609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.759053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.759525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:15.759525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.759525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:15.946068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:15.946642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:15.946727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:15.946789 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:15.947683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:15.948091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.948935 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:15.949817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:15.951187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.951759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:15.952412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:15.953346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:16.055950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:16.057375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:16.057520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:16.057589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:16.058399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:16.058925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.059862 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:16.060795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.061544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:16.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:16.063126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:16.063914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.064981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:16.112883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:16.113838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:16.114071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:16.114367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:16.115120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:16.115454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:16.115627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.116426 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.118385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:16.118811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.119153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:16.121126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:16.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:16.122833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:16.122833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:20.671290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:20.671860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:20.726594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:20.726594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:20.753783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:20.755342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:20.757106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:20.758896 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:20.759376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:20.759649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.760369 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.763609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.764365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:20.765037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.765037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:20.769976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.770432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.770969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:20.770969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.770969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:20.781628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:20.781950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.782801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_fdb_aging_settings failed after 0.111 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.783910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.784293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:20.784657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.784657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:20.784996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.785474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.785981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:20.785981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.785981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:20.947172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:20.947758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:20.947758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:20.947889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:20.948431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:20.948703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:20.950089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.950848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:20.951370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.951669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:20.952088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:20.952656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:21.057784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:21.058062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:21.058131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:21.058298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:21.059031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:21.059274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.059680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:21.061230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.062067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:21.062589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.063149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.063564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:21.063564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.063564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.064400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:21.065074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.067764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.067764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:21.067764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.067764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:21.113982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:21.114691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:21.114691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:21.114800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:21.115629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:21.116099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:21.116373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.117213 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.118298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:21.118701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.119026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:21.119314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:21.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:21.120248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:21.120248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:25.671913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:25.672424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:25.676035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:25.676035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:25.710933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:25.711261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.712325 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.714236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.714703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:25.714958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.714958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:25.715517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.715864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.716236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:25.716236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.716236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:25.755416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:25.755946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:25.756102 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:25.756872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:25.757158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:25.757285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.757840 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.759024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.759410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:25.759770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.759770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:25.760457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.761177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.761764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:25.761764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.761764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:25.948078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:25.948454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:25.948541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:25.948687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:25.949349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:25.949606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.950485 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:25.951208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.951667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.951914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:25.952275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.952631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:25.953064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:25.953730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:26.058405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:26.058990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:26.059180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:26.059180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:26.059760 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:26.060087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.060741 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:26.061703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.062362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.062764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:26.063320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:26.064192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:26.065232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.066180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:26.114652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:26.115257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:26.115356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:26.115551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:26.116548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:26.117330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:26.117733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.118767 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.119793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:26.120387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:26.120883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:26.121660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:26.122592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:26.122592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:26.122592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:30.672804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:30.674530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:30.674684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:30.675916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:30.716332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:30.716770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.717854 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.718939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.719534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:30.720188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:30.720971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.721840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:30.721840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.721840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.721840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:30.721840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.721840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:30.773236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:30.774260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:30.774454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:30.775359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:30.775541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:30.775685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:30.777622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:30.777968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.778279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:30.778279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.778279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.778279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:30.778279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.778279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:30.778599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.778952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.779315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:30.779315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.779315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:30.949094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:30.949094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:30.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:30.949394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:30.950210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:30.950586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.951496 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:30.952929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.953791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.954211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:30.954857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.956119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.956119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:30.956119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.956119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:30.956962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:30.957646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:30.958277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:30.958277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:30.958277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:30.958277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:31.059141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:31.060196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:31.060429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:31.060669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:31.061439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:31.061824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.062737 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:31.063909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.064636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.065099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:31.065619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:31.066276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:31.066896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.069333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:31.115448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:31.115724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:31.115932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:31.116025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:31.116582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:31.117576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:31.117576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.118197 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.121024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:31.121497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:31.121888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:31.122495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:31.123213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:31.123213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:31.123213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:35.674512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:35.676308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:35.676308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:35.676741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:35.678099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=471,neutron:UPDATE=2 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:37:35.716626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:35.717209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.718319 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.719441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.720055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:35.720611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:35.721359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:35.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:35.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.722735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:35.774405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:35.775125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:35.775351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:35.776409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:35.776959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:35.777227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:35.838425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:35.839915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.841171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.842549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.843634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:35.949434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:35.949654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:35.949724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:35.949835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:35.950861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:35.951087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.952049 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:35.952661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.953085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.953271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:35.953515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.953775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.953775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:35.953775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.953775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:35.953946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:35.954296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:35.954742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:35.954742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:35.954742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:35.954742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:36.060034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:36.060809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:36.061081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:36.061257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:36.062170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:36.062695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.063721 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:36.064773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.065259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.065717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:36.066096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.066472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.066472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:36.066472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.066472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.066872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:36.067262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.067717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.067717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:36.067717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.067717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:36.116267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:36.117096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:36.117096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:36.117096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:36.118064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:36.118541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:36.118780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.119831 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.120979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:36.121838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:36.122519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:36.123910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:36.124972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:36.124972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:36.124972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:40.675794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:40.676890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:40.676890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:40.677639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:40.721310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:40.721610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.722759 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.723641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.724075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:40.724469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.724469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:40.724857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.725384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.725950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:40.725950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.725950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:40.775667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:40.775667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:40.775801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:40.776706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:40.777137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:40.777414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.778067 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.779611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.780135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:40.780510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.780510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:40.780930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.781454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.781996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:40.781996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.781996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:40.949831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:40.951033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:40.951033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:40.951033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:40.951495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:40.951689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:40.954284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.954981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.956487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.957584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.958175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:40.958686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:40.959486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:41.060869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:41.061109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:41.061255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:41.061408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:41.062168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:41.062508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.065936 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:41.066935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.067379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.067913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:41.068297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.068855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.068855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:41.068855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.068855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.069018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:41.069700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.070292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.070292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:41.070292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.070292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:41.116921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:41.117139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:41.117298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:41.117421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:41.118112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:41.118407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:41.118577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.119227 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.120268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:41.120754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.121201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:41.121378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:41.122204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:41.122704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:41.122704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:45.676474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:45.677288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:45.677288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:45.677996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:45.710212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:45.710574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.711550 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.712453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.713088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:45.713697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:45.714463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.715388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:45.715388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.715388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.715388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:45.715388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.715388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:45.776037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:45.776893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:45.776893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:45.778125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:45.778514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:45.778734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.779759 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.781575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.782475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:45.783108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.783108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:45.783876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.784964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.785487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:45.785487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.785487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:45.951393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:45.951393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:45.951393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:45.951839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:45.951987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.952447 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:45.952981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.953721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:45.954214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.954630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:45.955041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:45.955658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:46.061362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:46.061716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:46.061904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:46.061980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:46.062676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.063786 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:46.064417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.064865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.065348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:46.065834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.066339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.066339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:46.066339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.066339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.066602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:46.067234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.067817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.067817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:46.067817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.067817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:46.117898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:46.119601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:46.119717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:46.119941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:46.120496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:46.120744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:46.120871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.121376 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.122416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:46.122860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.123266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:46.123545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:46.124146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:46.124623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:46.124623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:50.677363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:50.678567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:50.678567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:50.678971 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:50.721158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:50.721319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.721887 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:50.722907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:50.723522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:50.724093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:50.724894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:50.725686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:50.725686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:50.725686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:50.725686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:50.725686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.725686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:50.776864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:50.777695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:50.777810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:50.779920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:50.780530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:50.780812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.781773 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:50.783052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:50.783685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:50.784348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:50.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:50.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:50.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:50.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:50.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:50.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:50.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:50.999821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:51.000192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:51.000552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:51.000761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:51.001299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:51.001586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.002374 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:51.003095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.003501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.004030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:51.004348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.004684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.004684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:51.004684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.004684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.004794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:51.005163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.005606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.005606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:51.005606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.005606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:51.062106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:51.062267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:51.062387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:51.062495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:51.063312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:51.063749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.064797 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:51.065601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:51.067524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:51.068388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:51.068984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:51.069559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.070428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:51.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:51.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:51.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:51.119826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:51.122092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:51.123370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:51.123370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:51.124863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.126209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:51.127051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:51.127456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:51.128088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:55.677892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:55.678908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:55.678908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:55.678908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:55.714593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:55.715117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.716207 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:55.717197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:55.718142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:55.718555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:55.719421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:55.720044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:55.720044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:55.720044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:55.720044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:55.720044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.720044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:55.778234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:55.779227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:55.779530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:55.780880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:55.781188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:55.781412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.782172 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:55.783926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:55.784580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:55.785245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.785245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:55.785949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:55.786473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:55.787027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:55.787027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:55.787027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:56.000556 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:56.001126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:56.001215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:56.001767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:37:56.045295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:56.045295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.045824 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:56.047398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.047835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:56.048192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.048472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.048853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:56.049350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.049762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.049762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:56.049762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.049762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:56.062533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:56.062994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:56.063115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:56.063241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:56.064555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:56.064555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.064869 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:37:56.066233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:56.067095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:56.067698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.068300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.068811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:56.069382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.070240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.070240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:56.070240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.070240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:37:56.120462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:37:56.120668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:37:56.122143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:37:56.122143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:37:56.122569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:56.123203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:37:56.123306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.124091 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.125403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:37:56.125846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.126247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:37:56.126352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:37:56.126875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:37:56.127420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:37:56.127420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:00.678474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:00.679321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:00.679321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:00.680274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:00.718482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:00.718715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.719496 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:00.720923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:00.721593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:00.722285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:00.723047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:00.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:00.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:00.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:00.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:00.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.724039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:00.778910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:00.779830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:00.780045 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:00.781150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:00.781455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:00.781631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.782398 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:00.783991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:00.784518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:00.784923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.784923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:00.785279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:00.785884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:00.786680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:00.786680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:00.786680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:01.001524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:01.002780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:01.002780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:01.003100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:01.003764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:01.004017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.004763 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:01.006026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.006821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.007286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:01.007933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.008531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.008531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:01.008531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.008531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.008729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:01.009244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.009908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.009908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:01.009908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.009908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:01.128491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:01.129573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:01.129897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:01.130538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:01.130653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:01.130931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:01.131051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:01.131238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:01.132443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:01.132669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.133252 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:01.134540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:01.134947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:01.136693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.137212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.137652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:01.137652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.137652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:01.137652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.138740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:01.139279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.139979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.139979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:01.139979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.139979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:01.139979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.140279 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.140930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:01.141351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:01.141730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:01.142225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:01.142758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:01.142758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:01.142758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:05.679265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:05.680252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:05.680252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:05.681322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:05.725646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:05.725936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.726796 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:05.727673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:05.728118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:05.728550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:05.729328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:05.729890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:05.729890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:05.729890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:05.729890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:05.729890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.729890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:05.779911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:05.781312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:05.781446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:05.782421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:05.782743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:05.782888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.783628 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:05.784708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:05.785628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:05.786360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:05.787149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:05.788284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:05.788284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:05.788284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:05.788284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:05.788284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:05.788284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:06.002695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:06.003190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:06.003839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:06.003839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:06.004229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:06.004459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.005138 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:06.006491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.007374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.007760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:06.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.009132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.009132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:06.009132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.009132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.009491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:06.010278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.010941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.010941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:06.010941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.010941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:06.130032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:06.130356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:06.130798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:06.132072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:06.132376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:06.132517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:06.132803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:06.133078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:06.133573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:06.134041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.135092 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:06.136558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.137274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:06.138070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.138643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.139032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:06.139032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.139032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:06.139032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.140447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:06.140922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.141579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.141579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:06.141579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.141579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:06.141579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:06.141579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.142197 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.143615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:06.143978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.144311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:06.146168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:06.146676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:06.147147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:06.147147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:10.682801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:10.685973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:10.686608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:10.690226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:10.731826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:10.731979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Traceback (most recent call last):
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.732649 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:10.734365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:10.734663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:10.734981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.734981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:10.735530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:10.735882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:10.736798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:10.736798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.736798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:10.781203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:10.781759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:10.781759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:10.782820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:10.783220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.784884 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:10.785342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:10.785699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:10.786097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:10.786872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:10.787419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:10.787419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:10.787419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:10.787419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:10.787419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:10.787419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:11.004560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:11.005643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:11.005643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:11.005643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:11.006105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:11.006355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:11.010075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.011223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.012135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:11.013081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.013697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.013697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:11.013697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.013697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.014941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:11.015510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.015964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.015964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:11.015964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.015964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:11.130888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:11.131234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:11.131524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:11.131695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:11.131862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:11.132513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:11.133165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.133774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:11.133856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:11.134098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.134985 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:11.136710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:11.137684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:11.138607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.139090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.139460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:11.139460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.139460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:11.139460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.140185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:11.140646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.141410 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.141982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:11.142411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.142783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:11.143222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:11.143708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:11.144252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:11.144252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:15.685695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:15.686397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:15.686508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:15.688633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:15.726059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:15.726291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Traceback (most recent call last):
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.727242 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:15.728356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:15.729019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:15.729472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.729472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:15.729640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:15.730244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:15.730894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:15.730894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.730894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:15.781923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:15.782484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:15.782602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:15.783490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:15.783721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.785331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:15.786960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:15.787807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:15.788768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:15.790137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:16.005404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:16.006033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:16.006033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:16.006033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:16.006290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:16.006479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.006990 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:16.007636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.008152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:16.008545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.008915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:16.009294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.009928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:16.131940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:16.132883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:16.133041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:16.133268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:16.133930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:16.134601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:16.135108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.135500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.136184 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:16.136427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:16.137924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:16.139108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:16.143489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.144238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.144897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:16.144897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.144897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:16.144897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.145518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:16.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.147022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.147022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:16.147022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.147022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:16.147022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:16.147022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.147245 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.148742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:16.149296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.149724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:16.150282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:16.151075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:16.151725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:16.151725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:20.686608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:20.687299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:20.687376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:20.688757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:20.722656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:20.722944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.723777 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:20.724947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:20.725572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:20.725974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:20.726485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:20.727266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:20.727266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:20.727266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:20.727266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:20.727266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.727266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:20.782637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:20.783331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:20.783472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:20.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.785834 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:20.786666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:20.787227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:20.787777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.787777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:20.788084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:20.788769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:20.789579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:20.789579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:20.789579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:21.006381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:21.007360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:21.007472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:21.007553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:21.008147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:21.008313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.008832 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:21.009765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.010445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.011021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:21.011782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.012455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:21.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.013273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:21.133226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:21.133803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:21.133993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:21.133993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:21.134595 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:21.134830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.135156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.135676 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:21.136887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:21.137694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:21.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.138936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:21.139473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:21.139934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:21.140548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:21.140978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:21.140978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.141802 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.143577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:21.143970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.144335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:21.145456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:21.146750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:21.147594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:21.147594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:25.687220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:25.687783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:25.687933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:25.689334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:25.720978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:25.721296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.722036 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:25.722934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:25.723491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:25.723955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:25.724529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:25.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:25.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:25.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:25.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:25.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:25.783355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:25.784015 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:25.784278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:25.785587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:25.785836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:25.786018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.786665 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:25.787748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:25.788189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:25.788547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.788547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:25.788848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:25.789336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:25.789891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:25.789891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:25.789891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:26.007472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:26.008119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:26.008267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:26.008422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:26.009227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:26.009904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.010050 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:26.010914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.011275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.011891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:26.012304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.012584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.012584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:26.012584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.012584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.013666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:26.014058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.014426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.014426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:26.014426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.014426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:26.134442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:26.134804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:26.134931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:26.135289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:26.136570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:26.136570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.136658 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:26.137959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:26.138783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:26.139568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.140102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.140650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:26.140650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.140650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:26.140650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.141278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:26.141851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.142399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.142399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:26.142399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.142399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:26.142399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:26.142399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:26.143617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:26.143617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:26.143617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.143783 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.145873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:26.146274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.146595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:26.147275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:26.147877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:26.148519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:26.148519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:30.687740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:30.688636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:30.688636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:30.689799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:30.728132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:30.728406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.729214 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:30.730017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:30.730345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:30.730678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.730678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:30.730902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:30.731268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:30.731676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:30.731676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.731676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:30.784174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:30.784679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:30.784898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:30.785877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:30.786072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:30.786278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.786988 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:30.788454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:30.788921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:30.789368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.789368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:30.790237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:30.790802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:30.791394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:30.791394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:30.791394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:31.008466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:31.008907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:31.009020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:31.009155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:31.010092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:31.010433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.011219 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:31.012229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.012782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.013198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:31.013734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.014510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:31.014932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.015526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:31.135146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:31.135645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:31.135804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:31.135974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:31.136785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:31.137309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.138209 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:31.139871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:31.140340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:31.141537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.141922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.142324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:31.142324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.142324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:31.142324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.143118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:31.143703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.144241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.144241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:31.144241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.144241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:31.144241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:31.144241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:31.144931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:31.145038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:31.145308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.146038 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.147115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:31.147483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.147855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:31.148357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:31.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:31.149303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:31.149303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:35.688270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:35.688843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:35.688843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:35.689725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:35.691109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:38:35.726863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:35.727378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.728441 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:35.729409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:35.729691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:35.729929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.729929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:35.731039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:35.731365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:35.731697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:35.731697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.731697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:35.784583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:35.785268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:35.785531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:35.787057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:35.787664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:35.788094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.789123 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:35.789917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:35.790471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:35.790835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:35.791736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:35.792360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:35.792360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:35.792360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:35.792360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:35.792360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:35.792360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:36.009089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:36.009607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:36.009724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:36.009906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:36.010768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:36.011192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.012529 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:36.013422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.014504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:36.015435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.016294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:36.017355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.018590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:36.135620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:36.136068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:36.136154 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:36.136244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:36.136771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:36.137260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.138829 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:36.139904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.140401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:36.140944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:36.142335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.142739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.143171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:36.143171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.143171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:36.143171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:36.143171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:36.143171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:36.143475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.144922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:36.145437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.145977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.145977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:36.145977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.145977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:36.145977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:36.145977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.146655 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.147474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:36.147737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.147985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:36.148197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:36.148518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:36.148880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:36.148880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:40.688970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:40.689803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:40.690044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:40.691647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:40.718799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:40.719131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.720026 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:40.720887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:40.721272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:40.721598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.721598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:40.721669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:40.722064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:40.722546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:40.722546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.722546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:40.785198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:40.786189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:40.786189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:40.786453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:40.786710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:40.786873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.788144 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:40.788568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:40.789091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:40.789573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:40.790186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:40.790974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:40.790974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:40.790974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:40.790974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:40.790974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:40.790974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:41.009893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:41.010679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:41.010776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:41.010890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:41.011459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:41.011676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.012409 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:41.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.014724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:41.015397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.015970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:41.016608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.017383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:41.136382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:41.137481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:41.137650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:41.137762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:41.138698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:41.139085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.139867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:41.140802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:41.142116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:41.142786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:41.143398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.143958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:41.144755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:41.145656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:41.146061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:41.146061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.146845 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.148823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:41.149589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:41.150254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:41.151482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:41.153639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:41.153639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:41.153639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:45.689710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:45.690357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:45.690357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:45.691264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:45.727033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:45.727481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.728516 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:45.730258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:45.730695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:45.731166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.731166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:45.732245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:45.732698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:45.733242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:45.733242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.733242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:45.785870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:45.786469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:45.786577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:45.787375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:45.787567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:45.787718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.788328 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:45.789470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:45.790311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:45.790854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.790854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:45.791039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:45.791753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:45.792509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:45.792509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:45.792509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:46.010745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:46.010993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:46.011154 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:46.011279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:46.011653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:46.011871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.012415 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:46.013356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.014095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.014517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:46.015069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.015497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.015497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:46.015497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.015497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.015987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:46.016711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.017467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.017467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:46.017467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.017467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:46.137662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:46.138034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:46.138130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:46.138227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:46.138957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:46.139313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.139989 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:46.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:46.141470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:46.142092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:46.142653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:46.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.144138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:46.144749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:46.144749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:46.144749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:46.144749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:46.144749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:46.144982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:46.145252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.145850 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.147103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:46.147518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.147859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:46.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:46.149186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:46.149709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:46.149709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:50.690399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:50.691152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:50.691152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:50.692152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:50.729540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:50.729752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.730643 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:50.731631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:50.732105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:50.732530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.732530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:50.732904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:50.733438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:50.734065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:50.734065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.734065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:50.786656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:50.787341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:50.787445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:50.788378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:50.788591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:50.788718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.789926 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:50.791026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:50.791409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:50.791857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.791857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:50.792101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:50.792571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:50.793066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:50.793066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:50.793066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:51.011442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:51.011987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:51.011987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:51.011987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:51.012649 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:51.013038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.013852 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:51.015052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.015968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.016440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:51.016835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.017434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.017434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:51.017434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.017434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.018212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:51.018734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.019383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.019383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:51.019383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.019383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:51.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:51.138816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:51.138924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:51.138988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:51.139627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:51.139922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.140578 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:51.141378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:51.142717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:51.143748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:51.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:51.144518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.145079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:51.145672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:51.146525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:51.146525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:51.146525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:51.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:51.147404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:51.147649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:51.150445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.151021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:51.151389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:51.151814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:51.152342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:55.714903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:55.715782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:55.715782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:55.716621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:55.747751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:55.747992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.748786 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:55.749507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:55.750110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:55.750683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:55.751452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:55.752578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:55.752578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:55.752578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:55.752578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:55.752578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.752578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:55.787702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:55.788263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:55.788433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:55.789290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:55.789617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:55.789829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Traceback (most recent call last):
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.790664 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:55.791822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:55.792675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:55.793492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:55.794253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:55.795115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:55.795115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:55.795115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:55.795115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:55.795115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:55.795115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:56.012138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:56.012343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:56.012477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:56.012833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:38:56.013711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:56.013955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.014816 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:56.015907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.016423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.016951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:56.017471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.017950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.017950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:56.017950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.017950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:56.018858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.019675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.019675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:56.019675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.019675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:56.139200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:56.139712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:56.139852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:56.140020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:56.140787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:56.141316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.142159 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:38:56.143465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.143823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:56.144794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.145095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.145376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:56.145376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.145376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.146494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:56.146876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:38:56.147309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:38:56.148038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:56.148038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:38:56.148122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.148593 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.149348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:38:56.149620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.149956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:38:56.150292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:38:56.150681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:38:56.151158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:38:56.151158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:00.715592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:00.716272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:00.716400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:00.717432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:00.788529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:00.791309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:00.791539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:00.796580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:00.796855 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:00.796950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.801704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:00.801924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Traceback (most recent call last):
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.802453 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:00.803870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:00.804877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.805576 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:00.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:00.807554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.088 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:00.808602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:00.809230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:00.809768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:00.809768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.809768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:00.810129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:00.810614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:00.811193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:00.811193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:00.811193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:01.012861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:01.013776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:01.013776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:01.013776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:01.014656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:01.014847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.015354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:01.016147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.016729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.017504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:01.017768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.018073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.018073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:01.018073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.018073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.018281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:01.018627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.018991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.018991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:01.018991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.018991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:01.140078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.144034 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:01.144451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:01.144927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:01.145801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.146340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:01.146834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.147553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:01.148254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:01.148254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:01.148383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:01.148569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:01.149276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:01.149831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:01.150237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.151206 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.151886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:01.152443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.152794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:01.152981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:01.153354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:01.153680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:01.153680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:05.716484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:05.717469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:05.717469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:05.719736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:05.755180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:05.755418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.756102 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:05.756557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:05.756866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:05.757178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.757178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:05.757448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:05.757833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:05.758380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:05.758380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.758380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:05.791252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:05.791640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:05.791640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:05.792909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:05.793250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:05.793488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.794364 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:05.795480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:05.796277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:05.797042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:05.799443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:05.800498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:05.800498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:05.800498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:05.800498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:05.800498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:05.800498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:06.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:06.014308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:06.014395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:06.014499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:06.015029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:06.015312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.015937 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:06.017289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.018352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:06.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.019459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.019798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:06.020327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.020899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.020899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:06.020899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.020899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:06.140839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:06.141413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:06.141531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:06.141732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:06.142372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:06.142805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.143705 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:06.144924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.145733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:06.146327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.147064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:06.147677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:06.148572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:06.149700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.150661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:06.151240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:06.151609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:06.152305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:06.152797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:06.152797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:06.152797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:10.717053 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:10.717698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:10.717909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:10.719255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:10.752442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:10.752740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.753569 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:10.754838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:10.755441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:10.756099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:10.756831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:10.757942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:10.757942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:10.757942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:10.757942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:10.757942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.757942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:10.791691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:10.792215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:10.793101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:10.793448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:10.793836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:10.794096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.794899 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:10.796020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:10.796331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:10.796678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.796678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:10.797095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:10.797536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:10.798048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:10.798048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:10.798048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:11.014440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:11.014810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:11.015028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:11.015232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:11.016190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:11.016599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.017599 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:11.018924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.019453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:11.022324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.022643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.022643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:11.022643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.022643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.023653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:11.024195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.024752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.024752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:11.024752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.024752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:11.141514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:11.142224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:11.142334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:11.142476 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:11.143474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:11.144019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.144973 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:11.146076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.146425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:11.147534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.147849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.148123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:11.148123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.148123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.151951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:11.152480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:11.152969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:11.153255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:11.153255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.154359 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.155768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:11.156265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.156634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:11.157387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:11.157861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:11.158361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:11.158361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:15.717732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:15.718947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:15.719097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:15.720091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:15.755307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:15.755620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.756462 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:15.757511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:15.758442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:15.759099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:15.760105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:15.761460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:15.761460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:15.761460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:15.761460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:15.761460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.761460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:15.792557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:15.793155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:15.793457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:15.794665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:15.794866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:15.794984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.795603 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:15.796973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:15.797445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:15.797912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.797912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:15.798492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:15.799255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:15.799940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:15.799940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:15.799940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:16.015051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:16.015544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:16.015633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:16.015806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:16.016230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:16.016448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.017099 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:16.017726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.018227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.018646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:16.019045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.019440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.019440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:16.019440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.019440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:16.020056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.020559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.020559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:16.020559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.020559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:16.142394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:16.143266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:16.143266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:16.143626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:16.144029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:16.144805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.144885 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:16.145725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.146606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:16.146972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.147334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.147789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:16.148334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.148910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.148910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:16.148910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.148910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:16.151131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:16.151490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:16.151607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:16.151889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:16.152812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:16.152812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:16.152812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.153431 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.154430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:16.155401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:16.156056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:16.156630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:16.157201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:16.157201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:16.157201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:20.719020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:20.723056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:20.723056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:20.723056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:20.768117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:20.769096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.769436 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:20.770808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:20.771254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:20.772154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.772154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:20.773070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:20.773444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:20.773839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:20.773839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.773839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:20.793127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:20.793694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:20.793838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:20.795330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:20.795330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:20.795330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.796041 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:20.796987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:20.797490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:20.797840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.797840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:20.798240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:20.798708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:20.799298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:20.799298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:20.799298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:21.016519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:21.017192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:21.017354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:21.017517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:21.018589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:21.018951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.019925 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:21.021395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.023125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:21.023916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.024869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:21.025971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.027423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:21.143105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:21.143579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:21.143685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:21.144285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:21.144285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:21.144525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.144786 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:21.145551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.146056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:21.146467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:21.146851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:21.147225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.147735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:21.151550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:21.151896 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:21.151966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:21.152117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:21.152469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:21.152747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:21.152911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.153434 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.154216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:21.154629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:21.155031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:21.155537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:21.156090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:21.156090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:21.156090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:25.720591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:25.721465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:25.721465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:25.722412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:25.763133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:25.763435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.764231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:25.764909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:25.765380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:25.765771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:25.766212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:25.766760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:25.766760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:25.766760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:25.766760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:25.766760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.766760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:25.794088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:25.794536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:25.794656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:25.795360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:25.795564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:25.795714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.796299 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:25.797444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:25.797906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:25.798666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.798666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:25.799063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:25.800364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:25.801491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:25.801491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:25.801491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:26.017237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:26.017865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:26.018019 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:26.018229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:26.019028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:26.019406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.020304 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:26.021750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.022533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.023098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:26.023647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.024270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.024270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:26.024270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.024270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.024818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:26.025860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.027103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.027103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:26.027103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.027103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:26.143995 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:26.144471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:26.144708 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:26.144914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:26.145897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:26.146472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.147325 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:26.148543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.149056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:26.150054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.150388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.150663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:26.150663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.150663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.151710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:26.152106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.152524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.152524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:26.152524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.152524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:26.152524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:26.152681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:26.152833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:26.153089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:26.153579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:26.153892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:26.154080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.154606 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.156082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:26.156344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.156580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:26.157726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:26.158212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:26.158796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:26.158796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:30.721684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:30.722457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:30.722457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:30.723853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:30.761940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:30.762499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.763528 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:30.764592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:30.765338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:30.765928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:30.766453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:30.767049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:30.767049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:30.767049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:30.767049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:30.767049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.767049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:30.794802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:30.795393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:30.795579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:30.796776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:30.797134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:30.797428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.798355 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:30.799995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:30.800621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:30.801188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.801188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:30.802066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:30.803044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:30.803970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:30.803970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:30.803970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:31.060254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:31.060254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:31.060254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:31.060254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.060988 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:31.061781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.062993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:31.063541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.063984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:31.064419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.064959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:31.144662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:31.145265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:31.145429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:31.145571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:31.146195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:31.146595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.147432 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:31.148263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.148684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:31.149253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.149633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.149981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:31.149981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.149981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.150379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:31.150902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.151467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.151467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:31.151467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.151467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:31.152690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:31.153057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:31.153202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:31.153454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:31.153820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:31.154111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:31.154273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.154898 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.156400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:31.156762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.157105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:31.157931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:31.158602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:31.159159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:31.159159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:35.722524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:35.723385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:35.723621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:35.725249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:35.726940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:39:35.758388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:35.758839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.759802 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:35.761447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:35.761869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:35.762274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.762274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:35.763214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:35.763682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:35.764425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:35.764425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.764425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:35.795717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:35.796672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:35.796875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:35.798122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:35.798544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:35.798819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.799960 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:35.801570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:35.801985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:35.802337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.802337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:35.803258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:35.803719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:35.804481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:35.804481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:35.804481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:36.059957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:36.060433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:36.060584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:36.060748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:36.061687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:36.062096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:36.137383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.138367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.138960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.139357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.139672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.140067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.141822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:36.145281 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:36.147199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:36.147199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:36.147199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:36.147199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:36.147199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.147698 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:36.148836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.149386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:36.150270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.151791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:36.152502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:36.153412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:36.154393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:36.155226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:36.155295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:36.155485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.156034 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.156840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:36.157473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:36.157851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:36.158390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:36.158974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:36.158974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:36.158974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:40.723540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:40.724394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:40.724394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:40.725776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:40.759965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:40.760359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.761575 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:40.762630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:40.763267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:40.763864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:40.764602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:40.765558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:40.765558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:40.765558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:40.765558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:40.765558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.765558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:40.796532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:40.797392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:40.797611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:40.798736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:40.799178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:40.799478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.800492 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:40.802279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:40.802625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:40.802995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.802995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:40.804310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:40.805355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:40.805843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:40.805843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:40.805843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:41.060702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:41.061274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:41.061378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:41.061588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:41.062383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:41.062827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.063657 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:41.064871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.065939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:41.066918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.067592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:41.068214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.068952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:41.194282 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.194992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:41.195413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.195745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.196100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.196366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:41.196779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:41.197170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:41.197416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:41.198021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:41.198263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:41.198506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:41.198855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:45.724177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:45.724967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:45.724967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:45.726357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:45.758737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:45.758916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.759415 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:45.760223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:45.760800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:45.761355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:45.762115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:45.762924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:45.762924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:45.762924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:45.762924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:45.762924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.762924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:45.797031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:45.797534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:45.797692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:45.798460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:45.798693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:45.798837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.799467 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:45.800566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:45.801275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:45.801811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:45.802493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:45.803299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:45.803299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:45.803299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:45.803299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:45.803299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:45.803299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:46.061370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:46.061667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:46.061667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:46.061731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:46.062572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:46.062971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.064162 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:46.065116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.065892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.066473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:46.066925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.067389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.067389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:46.067389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.067389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.067615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:46.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.068790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.068790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:46.068790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.068790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:46.146678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:46.147314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:46.147477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:46.147721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:46.148289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:46.148650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.150084 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:46.150601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.151269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:46.151850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:46.152405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:46.153018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.153770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:46.194610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:46.195273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:46.195418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:46.195675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:46.196747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:46.197558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:46.197807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.198915 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.235030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:46.235378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.235609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:46.236686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:46.237260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:46.238013 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:46.238013 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:50.724767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:50.725492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:50.725492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:50.726297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:50.764109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:50.764629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.765423 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:50.766673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:50.767109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:50.767497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.767497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:50.767729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:50.768324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:50.768863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:50.768863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.768863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:50.797484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:50.797961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:50.798340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:50.799581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:50.799952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:50.800262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.801196 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:50.803152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:50.803456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:50.803696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.803696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:50.804643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:50.805158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:50.805511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:50.805511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:50.805511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:51.061959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:51.062557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:51.062708 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:51.062850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:51.063651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:51.063974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.064829 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:51.065708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.066723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.067160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:51.067771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.068442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.068442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:51.068442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.068442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.070100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:51.070972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.072140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.072140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:51.072140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.072140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:51.147344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:51.147610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:51.147712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:51.148067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:51.148587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:51.148984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.149436 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:51.150392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.150928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:51.151418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.151813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:51.153405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:51.153890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.154857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:51.195171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:51.195726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:51.195851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:51.196027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:51.196757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:51.197395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:51.197796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.198989 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.199945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:51.200317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.200583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:51.201253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:51.201790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:51.202192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:51.202192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:55.725575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:55.726288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:55.726494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:55.727731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:55.763659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:55.764126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.765101 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:55.766312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:55.766901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:55.767511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:55.768173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:55.768943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:55.768943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:55.768943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:55.768943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:55.768943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.768943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:55.798167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:55.798701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:55.799021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:55.799852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:55.800091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:55.800283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.800902 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:55.801958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:55.802399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:55.802733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.802733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:55.803065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:55.803517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:55.804101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:55.804101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:55.804101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:56.062776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:56.063352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:56.063441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:56.063566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:39:56.064185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:56.064406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.065199 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:56.066850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.067675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.069109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:56.069739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.070319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.070319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:56.070319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.070319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.071543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:56.072064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.072629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.072629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:56.072629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.072629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:56.148079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:56.148609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:56.148814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:56.149081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:56.150119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:56.150584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.151626 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:39:56.153306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.153777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:56.155086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.155460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.155820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:56.155820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.155820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.156782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:56.157278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.157784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.157784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:56.157784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.157784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:39:56.196199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:39:56.196790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:39:56.197082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:39:56.197168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:39:56.198191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:56.198683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:39:56.198947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.200318 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.201715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:39:56.202402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.203018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:39:56.204294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:39:56.205134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:39:56.206046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:39:56.206046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:00.726186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:00.727117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:00.727117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:00.737225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:00.767661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:00.767992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.768726 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:00.769793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:00.770570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:00.771309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:00.772165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:00.773435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:00.773435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:00.773435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:00.773435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:00.773435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.773435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:00.798835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:00.799366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:00.799561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:00.801114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:00.801428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:00.801601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.802313 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:00.803429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:00.804062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:00.804632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:00.805313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:00.806173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:00.806173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:00.806173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:00.806173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:00.806173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:00.806173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:01.063666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:01.064298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:01.064431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:01.064574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:01.065257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:01.065560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.067235 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:01.067987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.068910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:01.069748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.070580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:01.071566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.072468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:01.148707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:01.149241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:01.149396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:01.149810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:01.150336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:01.150674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.151303 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:01.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.152780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:01.153457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:01.154066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:01.154662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.155683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:01.196854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:01.197389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:01.197463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:01.197735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:01.198238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:01.198639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:01.198765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.199844 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.201446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:01.202130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:01.202711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:01.203445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:01.204442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:01.204442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:01.204442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:05.727402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:05.728553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:05.728661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:05.729387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:05.761223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:05.761550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.762318 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:05.763209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:05.763635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:05.764142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:05.764600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:05.765181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:05.765181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:05.765181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:05.765181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:05.765181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.765181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:05.800410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:05.800834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:05.800928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:05.804832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:05.805240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:05.805489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.806649 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:05.808314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:05.808621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:05.808852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.808852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:05.809809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:05.810152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:05.810531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:05.810531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:05.810531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:06.064609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:06.065598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:06.065598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:06.065598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:06.066438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:06.066845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.068182 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:06.069540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.070132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.070909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:06.071495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.071872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.071872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:06.071872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.071872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.073280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:06.073727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.075248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.075248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:06.075248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.075248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:06.152494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:06.152977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:06.153307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:06.153647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:06.154380 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:06.154598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.155213 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:06.156974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.158061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:06.158366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:06.158663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:06.158924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.159324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:06.198879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:06.199497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:06.199643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:06.199828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:06.200915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:06.200915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:06.200915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.202216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:06.202669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.203042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:06.203398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:06.204060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:06.204422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:06.204422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:10.728683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:10.729585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:10.729585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:10.730976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:10.766884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:10.767204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.768100 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:10.768858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:10.769477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:10.769874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:10.770380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:10.770962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:10.770962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:10.770962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:10.770962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:10.770962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.770962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:10.801109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:10.804400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:10.804554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:10.805435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:10.805687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:10.805892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.806563 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:10.808753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:10.809402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:10.809770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.809770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:10.811628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:10.812475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:10.813273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:10.813273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:10.813273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:11.065298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:11.065790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:11.065967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:11.065967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:11.066608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:11.066795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.067331 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:11.068254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.068920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.069320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:11.069868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.070598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:11.071224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.071933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:11.153749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:11.154353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:11.154491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:11.154665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:11.155226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:11.155695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.156609 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:11.157662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.158201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:11.158889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.159283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.159691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:11.159691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.159691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.160260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:11.160822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.161489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.161489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:11.161489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.161489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:11.199720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:11.200577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:11.200728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:11.201018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:11.201757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:11.202202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:11.202576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.203359 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.205366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:11.205811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.206670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:11.208649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:11.209196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:11.209772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:11.209772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:15.729631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:15.730476 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:15.730576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:15.731444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:15.772671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:15.772939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.774197 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:15.775448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:15.776262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:15.777134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:15.778242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:15.778988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:15.778988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:15.778988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:15.778988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:15.778988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.778988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:15.801803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:15.802113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:15.802220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:15.803025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:15.803412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:15.803700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.804661 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:15.805617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:15.806364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:15.806967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:15.807692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:15.808550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:15.808550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:15.808550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:15.808550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:15.808550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:15.808550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:16.066660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:16.067519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:16.067843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:16.067843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:16.068784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:16.069072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.069863 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:16.071594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.072341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.072984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:16.073590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.074201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.074201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:16.074201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.074201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.074772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:16.075300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.075850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.075850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:16.075850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.075850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:16.154563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:16.155171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:16.155313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:16.155711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:16.156450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:16.156868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.157724 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:16.158728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.159499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:16.159850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.160420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:16.160977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:16.161717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.162564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:16.200726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:16.201224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:16.201301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:16.201628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:16.202244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:16.202595 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:16.202802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.203405 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.204159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:16.204625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.205024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:16.205114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:16.205652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:16.206185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:16.206185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:20.730437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:20.731169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:20.731313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:20.732344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:20.771932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:20.771932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.772770 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:20.773924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:20.774314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:20.774633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.774633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:20.775048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:20.775752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:20.776473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:20.776473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.776473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:20.802381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:20.803103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:20.803367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:20.804350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:20.804736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:20.805110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.806266 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:20.808151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:20.808653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:20.809214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.809214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:20.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:20.810749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:20.811294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:20.811294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:20.811294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:21.067277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:21.067965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:21.067965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:21.068147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:21.068718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:21.069089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.069945 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:21.071277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.071733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.072610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:21.072994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.073266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.073266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:21.073266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.073266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.075061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:21.075624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.076217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.076217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:21.076217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.076217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:21.155187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:21.155408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:21.155491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:21.155587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:21.156327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:21.156743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.157584 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:21.158452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.158825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:21.159270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.159524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.159788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:21.159788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.159788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.160075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:21.160430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.160767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.160767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:21.160767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.160767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:21.201607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:21.202111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:21.202236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:21.202470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:21.203381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:21.203699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:21.203900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.204632 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.206195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:21.206609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.206952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:21.207672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:21.208213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:21.208737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:21.208737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:25.731151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:25.731715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:25.731715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:25.734104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:25.767451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:25.767767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.768591 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:25.769231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:25.769702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:25.770210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:25.770697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:25.771391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:25.771391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:25.771391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:25.771391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:25.771391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.771391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:25.803124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:25.803748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:25.804154 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:25.805073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:25.805339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:25.805729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.806558 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:25.807929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:25.808494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:25.808990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.808990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:25.809598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:25.810258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:25.810969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:25.810969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:25.810969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:26.067793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:26.068311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:26.069055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:26.069055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:26.069280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:26.069413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.070202 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:26.070799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.071676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:26.072394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.073069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:26.073806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.074790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:26.155600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:26.156023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:26.156122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:26.156229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:26.156841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:26.157153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.157840 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:26.159170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.159696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:26.160552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.160942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.161434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:26.161434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.161434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.162798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:26.163328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.163830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.163830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:26.163830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.163830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:26.202282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:26.202477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:26.202515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:26.202674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:26.203748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:26.204091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:26.204290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.205022 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.205821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:26.206230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.206566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:26.208634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:26.209323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:26.209799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:26.209799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:30.732249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:30.733174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:30.733332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:30.734271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:30.780591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:30.780974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.781299 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:30.782175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:30.782614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:30.783383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:30.783870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:30.784936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:30.784936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:30.784936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:30.784936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:30.784936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.784936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:30.803823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:30.804449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:30.804546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:30.805362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:30.805802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:30.806027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.806828 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:30.807694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:30.808519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:30.809244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:30.812262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:30.813632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:30.813632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:30.813632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:30.813632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:30.813632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:30.813632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:31.068538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:31.069274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:31.069371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:31.069500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:31.070279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:31.070662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.071447 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:31.072392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.072915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.073642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:31.074550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.075332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:31.075811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.076367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:31.156358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:31.157367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:31.157752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:31.157752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:31.158722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:31.159015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.159681 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:31.160482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.161064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:31.161333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.161817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:31.162283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:31.162805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.163405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:31.202823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:31.203507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:31.203744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:31.204063 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:31.204674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:31.205020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:31.205204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.205859 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.206835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:31.207667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:31.208281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:31.210200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:31.212051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:31.212051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:31.212051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:35.733250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:35.735287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:35.735287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:35.735287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:35.736706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:40:35.766174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:35.766513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.767513 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:35.769122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:35.769987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:35.770792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:35.771381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:35.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:35.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:35.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:35.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:35.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.772166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:35.804780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:35.805488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:35.805641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:35.807876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:35.808277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:35.808691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.809580 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:35.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:35.811185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:35.811728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:35.812429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:35.813269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:35.813269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:35.813269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:35.813269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:35.813269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:35.813269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:36.069526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:36.070260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:36.070479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:36.070612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:36.071418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:36.071629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.072291 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:36.073190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.074065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:36.074764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.075398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:36.076221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.077258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:36.156966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:36.157467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:36.157608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:36.157860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:36.158467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:36.158759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.159419 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:36.161362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.161915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:36.163172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.163573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.163984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:36.163984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.163984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.165653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:36.166343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.166913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.166913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:36.166913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.166913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:36.203689 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:36.204170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:36.204473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:36.204652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:36.205508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:36.206441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:36.206639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.208388 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.209572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:36.210213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.210842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:36.211072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:36.211648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:36.212249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:36.212249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:40.734093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:40.734748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:40.734748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:40.736467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:40.768899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:40.769247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.769971 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:40.770794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:40.771214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:40.771617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:40.772104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:40.772820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:40.772820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:40.772820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:40.772820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:40.772820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.772820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:40.805592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:40.806177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:40.806417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:40.807389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:40.807890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:40.808251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.809186 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:40.810586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:40.811310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:40.812058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.812058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:40.812337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:40.813344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:40.814329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:40.814329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:40.814329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:41.070277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:41.070805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:41.070936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:41.071127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:41.071972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:41.072443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.073411 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:41.074745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.075612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.076066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:41.076834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.077478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.077478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:41.077478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.077478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.077668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:41.078461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.079521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.079521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:41.079521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.079521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:41.157804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:41.158477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:41.158577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:41.158764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:41.159415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:41.159823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.161193 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:41.162882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.163425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:41.164342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.164822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:41.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.166241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:41.166862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.167464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.167464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:41.167464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.167464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:41.204421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:41.204869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:41.204969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:41.205267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:41.205876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:41.206304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:41.206557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.207388 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.209022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:41.209507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.209904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:41.210929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:41.211512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:41.212037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:41.212037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:45.734657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:45.736080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:45.736277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:45.738961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:45.788081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:45.789086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Traceback (most recent call last):
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.790036 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:45.793077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:45.793956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:45.794618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:45.795414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:45.796345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:45.796345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:45.796345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:45.796345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:45.796345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.796345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:45.806171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:45.806911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:45.807245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:45.810542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:45.810881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:45.811060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.812898 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:45.815693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:45.816485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:45.816765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.816765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:45.818393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:45.818918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:45.819477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:45.819477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:45.819477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:46.071036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:46.071660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:46.071808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:46.071959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:46.072860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:46.073423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.074257 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:46.075628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.076291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.077080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:46.077503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.077895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.077895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:46.077895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.077895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.078904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:46.079424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.080075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.080075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:46.080075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.080075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:46.158662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:46.159064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:46.159304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:46.159952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:46.160487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:46.161055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.162130 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:46.163050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.163888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:46.164376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.165089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:46.165795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:46.166768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.167684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:46.205087 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:46.205401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:46.205542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:46.205670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:46.206320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:46.206817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:46.207135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.208101 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.209092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:46.210040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:46.210641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:46.211524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:46.212396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:46.212396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:46.212396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:50.736351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:50.736840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:50.736992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:50.737820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:50.783047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:50.783528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.784197 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:50.785276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:50.785740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:50.786588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:50.787221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:50.787779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:50.787779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:50.787779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:50.787779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:50.787779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.787779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:50.807069 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:50.807445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:50.807609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:50.808411 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:50.808697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:50.808888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.809498 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:50.810373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:50.810693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:50.811044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.811044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:50.811313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:50.811733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:50.812277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:50.812277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:50.812277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:51.071728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:51.072310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:51.072508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:51.072865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:51.073541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:51.073958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.074913 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:51.076198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.076940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:51.078097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.078711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.078711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:51.078711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.078711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.079097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:51.079794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.080762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.080762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:51.080762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.080762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:51.159463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:51.160091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:51.160269 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:51.160479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:51.161246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:51.161534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.162188 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:51.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.164357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:51.165215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.165782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:51.166503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:51.167259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.168092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:51.205677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:51.206238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:51.206390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:51.206646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:51.207444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:51.207945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:51.208298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.209281 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.210305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:51.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:51.211595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:51.212371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:51.213230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:51.213230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:51.213230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:55.736985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:55.737604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:55.737683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:55.738540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:55.774017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:55.774376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.775204 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:55.776019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:55.776461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:55.776814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:55.777154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:55.777648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:55.777648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:55.777648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:55.777648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:55.777648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.777648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:55.807707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:55.808508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:55.808731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:55.809695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:55.810810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:55.810810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.811131 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:55.812452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:55.812742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:55.813020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.813020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:55.813843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:55.814655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:55.815129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:55.815129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:55.815129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:56.072600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:56.072929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:56.073047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:56.073125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:40:56.073550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:56.073784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.074364 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:56.075103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.076050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:56.076869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.077719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:56.079408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.081993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:56.160200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:56.160850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:56.160994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:56.160994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:56.161659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:56.161967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.162682 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:40:56.163630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.164694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:56.165241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:56.165687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:56.166145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.166781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:40:56.207038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:40:56.207492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:40:56.207605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:40:56.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:40:56.208507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:56.208822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:40:56.208992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.209714 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.211348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:40:56.211932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.212446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:40:56.212743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:40:56.213620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:40:56.214370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:40:56.214370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:00.738701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:00.739597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:00.740607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:00.743752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:00.784455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:00.784674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.785254 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:00.786162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:00.786618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:00.787052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.787052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:00.787136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:00.787665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:00.788278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:00.788278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.788278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:00.810914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:00.811762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:00.811854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:00.813256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:00.813598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:00.813799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.815145 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:00.816308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:00.816622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:00.816869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.816869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:00.817436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:00.818564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:00.818948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:00.818948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:00.818948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:01.073167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:01.073905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:01.073986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:01.074142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:01.075033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:01.075364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.076021 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:01.079388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.080236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.081187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:01.081829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.082524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.082524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:01.082524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.082524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.083025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:01.083605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.084323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.084323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:01.084323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.084323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:01.165078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:01.166737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:01.166845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:01.167799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:01.167799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:01.169132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.170903 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:01.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.172704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:01.172991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.173432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.173827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:01.173827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.173827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.174085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:01.174639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.175228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.175228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:01.175228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.175228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:01.207557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:01.208088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:01.208354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:01.208472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:01.209116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:01.209378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:01.209567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.210212 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.211797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:01.212937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.213351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:01.213478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:01.214420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:01.214978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:01.214978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:05.739935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:05.741181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:05.741521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:05.742986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:05.793162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:05.793387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.794467 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:05.795690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:05.796273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:05.796841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.796841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:05.797465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:05.798789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:05.800789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:05.800789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.800789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:05.812133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:05.812219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:05.813323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:05.813779 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:05.814070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.814983 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:05.815853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:05.816250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:05.816626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.816626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:05.816710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:05.817287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:05.817978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:05.817978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:05.817978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:06.073792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:06.074431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:06.074489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:06.074584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:06.075059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:06.075277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.075801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:06.076395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.076947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:06.077463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.077890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:06.078340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.078913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:06.166728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:06.167119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:06.167119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:06.167230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:06.168372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:06.169310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.169801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:06.170541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.171188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:06.171429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.171818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.172225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:06.172225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.172225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.172823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:06.173353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.173878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.173878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:06.173878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.173878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:06.208437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:06.209041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:06.209135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:06.209324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:06.210142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:06.210738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:06.210890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.212086 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.213070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:06.214078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:06.214406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:06.214879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:06.215399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:06.215399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:06.215399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:10.740720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:10.741545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:10.741545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:10.742867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:10.780439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.781953 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:10.782745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:10.783507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:10.783849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:10.784372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:10.784897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:10.784897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:10.784897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:10.784897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:10.784897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.784897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:10.812420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:10.814027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:10.814233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:10.817080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:10.817080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:10.817080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.817897 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:10.819551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:10.820275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:10.820893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:10.821570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:10.823307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:10.823307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:10.823307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:10.823307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:10.823307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:10.823307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:11.075584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:11.076145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:11.076289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:11.076449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:11.077269 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:11.077518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.078225 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:11.079557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.080124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.081391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:11.081851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.082337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.082337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:11.082337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.082337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.084269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:11.085336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.085952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.085952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:11.085952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.085952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:11.167341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:11.167815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:11.167815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:11.167920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:11.169025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:11.169154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.169890 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:11.170876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:11.172178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.172899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:11.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:11.174130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.174726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:11.209100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:11.209553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:11.209622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:11.209903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:11.210728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:11.211219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:11.211508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.212593 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.213899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:11.214566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:11.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:11.215641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:11.216605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:11.216605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:11.216605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:15.741427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:15.742299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:15.742299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:15.743274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:15.781535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:15.781821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.782473 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:15.783521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:15.784059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:15.784557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:15.785336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:15.786063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:15.786063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:15.786063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:15.786063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:15.786063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.786063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:15.813110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:15.813685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:15.813991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:15.814957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:15.815807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:15.815807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.816030 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:15.817546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:15.817853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:15.818157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.818157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:15.819384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:15.819871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:15.820382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:15.820382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:15.820382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:16.076481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:16.077274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:16.077340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:16.077458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:16.078174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:16.078397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.078972 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:16.079874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.080382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.080898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:16.081341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.081732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:16.082235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.082783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:16.168048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:16.168884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:16.169148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:16.169353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:16.170225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:16.170460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.171112 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:16.172625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.173330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:16.174173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.174727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.175330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:16.175330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.175330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.176032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:16.176743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.177519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.177519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:16.177519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.177519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:16.209709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:16.210445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:16.210445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:16.210560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:16.211125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:16.211638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:16.211887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.212691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.215734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:16.216343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:16.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:16.217496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:16.218278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:16.218278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:16.218278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:20.742065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:20.742931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:20.743088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:20.744650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:20.789712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:20.790031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.790899 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:20.791998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:20.792441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:20.792921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.792921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:20.793390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:20.793937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:20.794543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:20.794543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.794543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:20.814179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:20.814937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:20.815281 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:20.816400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:20.816598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:20.816748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.818160 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:20.820213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:20.820910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:20.821557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.821557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:20.822795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:20.823777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:20.824519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:20.824519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:20.824519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:21.077355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:21.077707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:21.077756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:21.077839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:21.078770 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:21.079052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.079712 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:21.080453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.080978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.081270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:21.081677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.082140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:21.082607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.083194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:21.168824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:21.169074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:21.169145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:21.169218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:21.170044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:21.170311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.170936 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:21.171889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.172414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:21.172657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.173057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:21.173504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:21.174095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.174638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:21.210317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:21.210586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:21.210676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:21.210831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:21.211742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:21.212046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:21.212202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.213163 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.214812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:21.215127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.215403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:21.216452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:21.217071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:21.217706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:21.217706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:25.742683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:25.743193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:25.743193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:25.746524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:25.775195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:25.775508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.776412 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:25.777290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:25.777607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:25.777863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.777863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:25.778077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:25.778611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:25.778970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:25.778970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.778970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:25.814830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:25.815518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:25.815726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:25.818231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:25.818423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:25.818728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.819765 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:25.821483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:25.822126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:25.822702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.822702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:25.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:25.824350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:25.825702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:25.825702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:25.825702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:26.079170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:26.079170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:26.079170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:26.079170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:26.079841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:26.080237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.081183 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:26.082481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.083290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.084148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:26.084971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.085797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:26.086509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.087368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:26.170104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:26.170955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:26.170955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:26.170955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:26.171894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:26.172421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.173374 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:26.174501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.175623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:26.176279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:26.176861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:26.177475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.178180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:26.210868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:26.211653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:26.211772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:26.212067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:26.213205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:26.213645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:26.213894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.214866 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.216505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:26.217368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:26.217936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:26.218692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:26.219527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:26.219527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:26.219527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:30.743352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:30.744074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:30.744074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:30.745308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:30.795784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:30.796993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.797879 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:30.799216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:30.799818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:30.800398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:30.801147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:30.801922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:30.801922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:30.801922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:30.801922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:30.801922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.801922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:30.815805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:30.816543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:30.816643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:30.817745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:30.818085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:30.818269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.819108 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:30.820160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:30.820815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:30.821479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:30.821970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:30.822506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:30.822506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:30.822506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:30.822506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:30.822506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:30.822506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:31.079205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:31.079773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:31.079896 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:31.079958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:31.081516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:31.081869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.082800 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:31.083911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.084544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.084799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:31.085311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.085729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:31.086260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.086828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:31.170823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:31.171582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:31.171658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:31.172180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:31.172593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:31.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.174047 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:31.174960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.175604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:31.175914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.176304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.176700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:31.176700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.176700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.176914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:31.177523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.178062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.178062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:31.178062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.178062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:31.211626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:31.212436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:31.212648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:31.212913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:31.213730 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:31.214165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:31.214407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.215243 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.216461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:31.216737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.217037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:31.217608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:31.218675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:31.219030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:31.219030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:35.744131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:35.744575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:35.744662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:35.745398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:35.746800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:41:35.780180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:35.780439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Traceback (most recent call last):
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.781309 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:35.782765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:35.783659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:35.784325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:35.785103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:35.786311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:35.786311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:35.786311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:35.786311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:35.786311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.786311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:35.817063 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:35.818473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:35.818473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:35.819346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:35.819636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:35.819796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:35.824312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:35.824805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.825195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:35.825590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:35.826141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:36.080103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:36.082031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:36.082031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:36.082031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:36.082031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:36.082031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.082759 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:36.084310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.084711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.085850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:36.086335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.086605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.086605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:36.086605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.086605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.087708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:36.088160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.088529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.088529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:36.088529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.088529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:36.171831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:36.172555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:36.172703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:36.172872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:36.173714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:36.174098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:36.176238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.176952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.177513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:36.177918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.178332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.178761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:36.179402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.180014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.180014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:36.180014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.180014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:36.212402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:36.213155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:36.215073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:36.215073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:36.215073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:36.215073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:36.215356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.216292 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.217939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:36.218408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.218805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:36.219253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:36.219777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:36.220412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:36.220412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:40.744796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:40.745956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:40.746142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:40.746746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:40.781950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:40.782148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Traceback (most recent call last):
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.782596 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:40.783352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:40.783966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:40.784643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.784643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:40.784754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:40.785496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:40.786249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:40.786249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.786249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:40.817794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:40.818211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:40.818391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:40.819492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:40.819617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:40.819697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.820348 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:40.822168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:40.822851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:40.823441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.823441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:40.824132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:40.824977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:40.825942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:40.825942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:40.825942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:41.080670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:41.081319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:41.081450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:41.081605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:41.082529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:41.082842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.083502 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:41.084743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.085545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:41.086326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.087101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:41.087713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.088559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:41.172952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:41.173276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:41.173554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:41.173812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:41.174928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:41.175408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.176378 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:41.177780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.178296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:41.179178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.179544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.179928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:41.179928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.179928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.180830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:41.181341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.181862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.181862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:41.181862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.181862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:41.213035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:41.213579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:41.213710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:41.213903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:41.214424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:41.214695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:41.214871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.215538 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.216655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:41.217599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:41.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:41.219287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:41.219911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:41.219911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:41.219911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:45.493514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:45.494472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:45.494472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:45.504490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 02:41:45.504627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 02:41:45.504627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:41:45.543219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.543575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.544104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.544967 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.546183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.546573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.546969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.546969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.546969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.546969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Fixing resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:41:45.581869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, resource={'id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['97900556-2eeb-4240-89bd-ed8258b9022d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:48Z', 'updated_at': '2026-04-13T02:30:49Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9202}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.582278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.582696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8120b9cd-5d7c-4d4d-9a57-22a6c716f312, external_ids_update=None, columns={'tag': 9202}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.582840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.583816 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Failed to fix resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.585236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.585662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.586048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.586048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.586048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.586048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:41:45.615384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.615772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.616145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.616298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.617169 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.618560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.619596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:41:45.657315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.658034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.658503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.658742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.659835 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.661555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.663029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.664229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.664229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.664229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.664229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Fixing resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:41:45.746186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:45.747725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:45.749338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:45.752663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:45.780446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, resource={'id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'enable_snat': True}, 'gw_port_id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-exgz1', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'ip_address': '172.24.5.236', 'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, {'ip_address': '2001:db8::2c9', 'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 3, 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46'}, resource_type=routers, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.780729 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.780979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.781189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.781314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.783161 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.783456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Unable to update router 530b037d-a47e-4f63-bd55-041139eeee2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Failed to fix resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 13 02:41:45.786527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.786928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.787354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Fixing resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:41:45.861721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:45.864706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:45.866134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:45.867496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.871761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.876219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.877315 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.884116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.884686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.885118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.885118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:45.886649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.887060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.887500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:45.887500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.887500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:45.918037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:45.918719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Traceback (most recent call last):
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.920107 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task check_fdb_aging_settings failed after 0.172 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.923557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.924164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:45.924698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.924698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:45.925770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:45.926337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:45.926829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:45.926829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:45.926829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:46.007834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, resource={'id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'name': '', 'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'project_id': '', 'mac_address': 'fa:16:3e:92:a4:24', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000165409', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.008171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.236/24', '2001:db8::2c9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.008517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3ec69888-b9c9-4830-9d48-853dd81cf68b, lrouter_port=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.008776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.009696 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Maintenance task: Failed to fix resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.010819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.011218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.011586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:46.011586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.011586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:46.011586 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task check_for_inconsistencies finished in 0.516 seconds
Apr 13 02:41:46.081461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:46.082262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:46.082262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:46.082262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:46.083088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.083345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.084026 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:46.084917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.086186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:46.086942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.087671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:46.092189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.093586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:46.173785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:46.174169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:46.174169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:46.174304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:46.175099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.175463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.176101 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:46.177075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.177787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.178138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.178672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.179269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:46.179269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.179269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.179410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:46.180411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.181222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.181222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:46.181222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.181222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:46.213498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:46.214159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:46.214293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:46.214538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:46.215067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.215421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:46.215632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.216375 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.217654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:46.218112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.218474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:46.219138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:46.219694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:46.220291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:46.220291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:50.747757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:50.748757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:50.748757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:50.749485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:50.782491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:50.783208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.783690 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:50.784686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:50.785413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:50.786142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:50.786954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:50.787985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:50.787985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:50.787985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:50.787985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:50.787985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.787985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:50.919870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:50.919959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:50.920330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:50.921517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:50.921894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:50.921950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Traceback (most recent call last):
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.922580 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:50.923384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:50.923672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:50.923937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.923937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:50.924171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:50.924531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:50.924927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:50.924927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:50.924927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:51.082153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:51.082837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:51.083034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:51.083227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:51.083954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:51.084159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.084690 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:51.085747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.086663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.086974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:51.087523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.088055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:51.088597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.089125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:51.174540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:51.175082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:51.175196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:51.175394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:51.176300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:51.176764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.177727 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:51.179158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.179927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:51.180300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.180874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.181806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:51.181806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.181806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.182032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:51.182544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.183036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.183036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:51.183036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.183036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:51.214343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:51.214933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:51.215060 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:51.215223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:51.215745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:51.216152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:51.216391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.217155 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.218688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:51.219384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.219958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:51.221555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:51.222578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:51.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:51.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:55.749980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:55.751285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:55.751285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:55.751687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:55.784404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:55.784769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.785968 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:55.787471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:55.788138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:55.788771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:55.789560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:55.790593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:55.790593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:55.790593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:55.790593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:55.790593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.790593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:55.920633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:55.921747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:55.983073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:55.983073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:55.983073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:55.983073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Traceback (most recent call last):
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:55.987449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:55.988538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-5b6b3633-46e3-457d-968f-cc5f175dceb3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.988851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:55.989194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:55.989641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:56.082757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:56.083559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:56.083790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:56.083899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:41:56.084504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:56.084802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.085690 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:56.087298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.088254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.088673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:56.089285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.089943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.089943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:56.089943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.089943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.090621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:56.091180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.091761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.091761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:56.091761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.091761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:56.175571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:56.176301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:56.176418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:56.176620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:56.177444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:56.178122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.179045 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:41:56.180417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.180935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:56.181291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.181648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.181994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:56.181994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.181994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.182543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:56.182996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.183501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.183501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:56.183501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.183501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:41:56.215118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:41:56.215439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:41:56.215539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:41:56.215763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:41:56.216537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:56.217056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:41:56.217372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.218291 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.219370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:41:56.219676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.219935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:41:56.220442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:41:56.220799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:41:56.221210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:41:56.221210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:00.750855 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:00.751677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:00.751886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:00.753301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:00.789276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:00.789676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.790821 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:00.791957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:00.792437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:00.792809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.792809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:00.793157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:00.793809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:00.794342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:00.794342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.794342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:00.921896 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:00.923239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:00.923239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:00.924357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:00.924836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:00.925122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.926125 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:00.927270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:00.928132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:00.928788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:00.930398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:00.931503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:00.931503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:00.931503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:00.931503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:00.931503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:00.931503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:01.083440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:01.083957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:01.084195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:01.084396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:01.085509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:01.085915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.087074 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:01.088698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.089421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.090021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:01.090746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.091332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.091332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:01.091332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.091332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.091634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:01.092195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.092795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.092795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:01.092795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.092795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:01.176272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:01.176956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:01.177126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:01.177316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:01.178248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:01.178813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.179824 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:01.181322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.182413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:01.182799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.183441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.184126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:01.184126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.184126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.184657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:01.185535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.186582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.186582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:01.186582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.186582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:01.215950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:01.216748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:01.216907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:01.217193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:01.218247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:01.218703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:01.218962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.220203 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.221848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:01.222375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.222742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:01.223388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:01.223983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:01.224508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:01.224508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:05.751568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:05.752505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:05.752624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:05.753991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:05.798199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:05.798829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.800112 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:05.801350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:05.801773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:05.802228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.802228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:05.802692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:05.803402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:05.804156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:05.804156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.804156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:05.923422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:05.924713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:05.924911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:05.927241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:05.927718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:05.928209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.930558 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:05.931828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:05.932430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:05.932802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.932802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:05.932990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:05.933493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:05.934026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:05.934026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:05.934026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:06.084232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:06.084775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:06.084926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:06.085132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:06.085931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:06.086267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.087074 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:06.089203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.090271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:06.091062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.091711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:06.092416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.093620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:06.176920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:06.177529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:06.177731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:06.177888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:06.179208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:06.179208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.180119 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:06.181595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.182560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:06.183112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.183830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:06.184582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:06.185438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.186616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:06.216833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:06.217347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:06.217606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:06.217812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:06.218252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:06.218716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:06.218716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.219519 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.220777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:06.221560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.222302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:06.222634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:06.223306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:06.223847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:06.223847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:10.752687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:10.753547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:10.753547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:10.755021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:10.793321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:10.793666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.794459 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:10.795201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:10.795695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:10.796434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:10.797317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:10.798455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:10.798455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:10.798455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:10.798455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:10.798455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.798455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:10.924973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:10.925765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:10.925884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:10.927026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:10.927316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:10.927485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.928128 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:10.929179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:10.929777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:10.930401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:10.931268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:10.932113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:10.932113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:10.932113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:10.932113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:10.932113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:10.932113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:11.085025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:11.085720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:11.085857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:11.086024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:11.087139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:11.087481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.088437 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:11.089809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.090553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.091123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:11.091661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.092303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.092303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:11.092303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.092303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.092722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:11.093592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.094561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.094561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:11.094561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.094561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:11.177965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:11.178597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:11.178675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:11.178816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:11.179650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:11.180105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.180893 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:11.181928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.182668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:11.182990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.183573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.184229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:11.184229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.184229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.184517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:11.185268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.186220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.186220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:11.186220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.186220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:11.217433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:11.217985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:11.218089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:11.218280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:11.218857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:11.219285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:11.219528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.220664 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.221632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:11.222364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:11.222942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:11.223720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:11.224619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:11.224619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:11.224619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:15.753401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:15.754521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:15.754792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:15.756124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:15.795752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:15.796128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.797125 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:15.798283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:15.799042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:15.799759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:15.800630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:15.801602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:15.801602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:15.801602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:15.801602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:15.801602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.801602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:15.925904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:15.927040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:15.927385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:15.928696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:15.929184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:15.929454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.930576 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:15.932353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:15.932719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:15.933116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.933116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:15.934471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:15.935128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:15.935640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:15.935640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:15.935640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:16.085895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:16.086825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:16.086825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:16.086825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:16.087866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:16.088261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.089458 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:16.090946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.091550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.092328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:16.092862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.093307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.093307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:16.093307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.093307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.094124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:16.094653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.095488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.095488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:16.095488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.095488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:16.178881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:16.179642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:16.179733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:16.179933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:16.180508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:16.180819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.181559 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:16.182431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.182887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:16.183302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.183645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.183988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:16.183988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.183988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.184565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:16.185064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.185735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.185735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:16.185735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.185735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:16.218094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:16.218801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:16.218949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:16.219285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:16.220280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:16.220760 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:16.221042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.222257 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.223968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:16.224538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.224940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:16.225457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:16.226599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:16.227414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:16.227414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:20.754375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:20.755386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:20.755386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:20.756819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:20.797758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:20.798162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.798631 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:20.799640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:20.800545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:20.801214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:20.802070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:20.802981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:20.802981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:20.802981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:20.802981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:20.802981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.802981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:20.926990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:20.927838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:20.928065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:20.929351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:20.929640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:20.929894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.930899 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:20.931776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:20.932169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:20.932541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:20.932977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:20.933545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:20.933545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:20.933545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:20.933545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:20.933545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:20.933545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:21.086635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:21.087286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:21.087502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:21.087686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:21.088347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:21.088970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.089880 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:21.091169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.092036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.092587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:21.093288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.093943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.093943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:21.093943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.093943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.094437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:21.095384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.096129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.096129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:21.096129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.096129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:21.179774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:21.180269 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:21.180487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:21.180595 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:21.181442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:21.181872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.182621 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:21.184578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.185330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:21.186100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.186652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.187257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:21.187257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.187257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.187901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:21.188421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.188933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.188933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:21.188933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.188933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:21.218576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:21.226888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:21.227036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:21.227196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:21.227789 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:21.228462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:21.228462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.229180 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.230208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:21.231126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:21.231879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:21.232398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:21.232769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:21.232769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:21.232769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:25.755190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:25.756272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:25.756272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:25.756555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:25.793486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:25.793975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.795326 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:25.796359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:25.796956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:25.797583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:25.798286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:25.799104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:25.799104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:25.799104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:25.799104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:25.799104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.799104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:25.928229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:25.929045 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:25.929659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:25.931111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:25.931543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:25.931840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.933056 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:25.935375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:25.936119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:25.936719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.936719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:25.937988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:25.938690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:25.939489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:25.939489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:25.939489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:26.087718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:26.088159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:26.088615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:26.088745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:26.089507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:26.089902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.090928 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:26.091914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.092760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.093175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:26.093741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.094429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:26.095106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.096050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:26.180461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:26.180655 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:26.180733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:26.180840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:26.181334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:26.181674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.182295 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:26.182997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.183914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:26.184803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:26.185545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:26.186339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.187316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:26.226948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:26.227664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:26.227857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:26.228302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:26.228964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:26.229494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:26.229752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.230873 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.232482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:26.233165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:26.233835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:26.234747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:26.235703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:26.235703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:26.235703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:30.755787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:30.756890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:30.756890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:30.757082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:30.795929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:30.796648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.797800 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:30.798689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:30.799370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:30.800005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:30.800838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:30.802876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:30.802876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:30.802876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:30.802876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:30.802876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.802876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:30.956014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:30.956014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:30.956014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:30.956014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:30.956014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:30.956014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:30.956673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:30.957644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.958374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:30.959343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:30.960242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:31.088775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:31.089319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:31.089513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:31.089513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:31.090638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:31.091088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.092410 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:31.093962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.095745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:31.096548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.097199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.097652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:31.104373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.105411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.105411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:31.105411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.105411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:31.181022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:31.183954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:31.183954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:31.183954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:31.183954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:31.183954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.184222 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:31.185433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.185972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:31.186330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.186666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.187092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:31.187092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.187092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.187418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:31.187920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.188456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.188456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:31.188456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.188456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:31.228164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:31.228717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:31.228856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:31.229167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:31.229799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:31.230322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:31.230938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.231847 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.233569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:31.234110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.234561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:31.235237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:31.235804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:31.236347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:31.236347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:35.756521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:35.757417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:35.757690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:35.759170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:35.761424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=434 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:42:35.800524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:35.800969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.802142 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:35.803473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:35.804082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:35.804716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:35.805507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:35.806344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:35.806344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:35.806344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:35.806344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:35.806344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.806344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:35.930151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:35.930972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:35.931153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:35.932254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:35.932749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:35.933031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Traceback (most recent call last):
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.934289 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:35.935982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:35.936616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:35.937120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.937120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:35.937202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:35.937670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:35.938249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:35.938249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:35.938249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:36.089580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:36.090312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:36.090393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:36.090513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:36.091567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:36.091836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.092514 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:36.093543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.094260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.094639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:36.095210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.095794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:36.096449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.097324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:36.181744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:36.182423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:36.182566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:36.182724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:36.183261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:36.183555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.184218 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:36.185389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.186184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:36.186790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.187395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:36.188048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:36.188827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.189719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:36.228900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:36.229614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:36.229720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:36.230019 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:36.231155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:36.231568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:36.231815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.232812 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.234226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:36.234858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.235484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:36.235661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:36.236397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:36.237195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:36.237195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:40.757268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:40.757992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:40.757992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:40.758984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:40.793851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:40.794598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.795508 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:40.796449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:40.796847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:40.797301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:40.797945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:40.798473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:40.798473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:40.798473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:40.798473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:40.798473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.798473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:40.931148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:40.931643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:40.931881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:40.932931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:40.933293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:40.933532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.934454 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:40.936186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:40.936976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:40.937768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.937768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:40.938061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:40.938824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:40.939516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:40.939516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:40.939516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:41.090596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:41.090967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:41.091197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:41.091368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:41.092437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:41.092794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.093810 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:41.095574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.096267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.097060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:41.097639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.098292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.098292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:41.098292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.098292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.098848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:41.099643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.100571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.100571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:41.100571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.100571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:41.182528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:41.182894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:41.182988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:41.183229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:41.183791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:41.184259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.185027 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:41.186548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.187508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:41.188168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.188874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:41.189806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:41.190821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.192109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:41.229558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:41.229922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:41.230108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:41.230348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:41.230797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:41.231230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:41.231526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.232638 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.234654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:41.235612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.236538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:41.236987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:41.237573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:41.238162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:41.238162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:45.758165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:45.761401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:45.761401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:45.761401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:45.795940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:45.796473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.797786 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:45.802481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:45.803338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:45.804336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:45.805203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:45.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:45.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:45.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:45.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:45.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.806240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:45.931962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:45.932686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:45.932832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:45.934429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:45.934838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:45.935366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.936374 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:45.937553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:45.938209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:45.938714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:45.939434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:45.940377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:45.940377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:45.940377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:45.940377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:45.940377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:45.940377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:46.091403 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:46.091930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:46.092025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:46.092149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:46.093105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:46.093450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.094352 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:46.095382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.096082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.096438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:46.096787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.097205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.097205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:46.097205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.097205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.097314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:46.097767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.098800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.098800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:46.098800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.098800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:46.183276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:46.183579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:46.183765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:46.184307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:46.184931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:46.185379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.186231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:46.187303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.188047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:46.188387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.188997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:46.189627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:46.190562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.191384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:46.230226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:46.230598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:46.230742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:46.230980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:46.232022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:46.232516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:46.232833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.233860 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.235388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:46.236097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.236727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:46.237137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:46.237973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:46.238844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:46.238844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:50.758926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:50.759666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:50.759778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:50.761035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:50.796580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:50.797168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.797985 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:50.798804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:50.799413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:50.800432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:50.801169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:50.802501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:50.802501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:50.802501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:50.802501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:50.802501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.802501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:50.932690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:50.933331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:50.933539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:50.934632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:50.934972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:50.935245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.936260 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:50.937859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:50.938528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:50.939128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.939128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:50.939747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:50.940268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:50.940821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:50.940821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:50.940821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:51.092330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:51.092915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:51.093074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:51.093242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:51.093935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:51.094755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.095702 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:51.096957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.097482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.098239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:51.098651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.099042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.099042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:51.099042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.099042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.099802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:51.100309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.100817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.100817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:51.100817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.100817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:51.183974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:51.184345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:51.184474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:51.184640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:51.185378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:51.186179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.186375 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:51.187664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.188135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:51.189125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.189468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.189820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:51.189820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.189820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.190912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:51.191407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.191905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.191905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:51.191905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.191905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:51.231222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:51.232157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:51.232318 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:51.232643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:51.233312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:51.233733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:51.233986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.234881 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.236090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:51.236864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:51.237665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:51.238823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:51.239657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:51.239657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:51.239657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:55.760041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:55.760863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:55.760863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:55.762159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:55.797844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:55.798314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.799449 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:55.800580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:55.801328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:55.802059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:55.802957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:55.803692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:55.803692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:55.803692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:55.803692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:55.803692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.803692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:55.933724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:55.934845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:55.935310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:55.936537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:55.936929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:55.937215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.938203 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:55.940073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:55.940646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:55.941238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.941238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:55.941795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:55.942430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:55.942982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:55.942982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:55.942982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:56.093341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:56.094279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:56.094748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:56.094748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:42:56.095634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:56.096256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.097209 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:56.098845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.099362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.100674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:56.101070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.101458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.101458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:56.101458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.101458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.102406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:56.103070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.103626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.103626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:56.103626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.103626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:56.184722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:56.185466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:56.185631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:56.185847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:56.186658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:56.187167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.188098 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:42:56.189508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.190204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:56.190940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.191544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.192148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:56.192148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.192148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.192755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:56.193577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.194382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.194382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:56.194382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.194382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:42:56.232321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:42:56.233151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:42:56.233314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:42:56.233653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:42:56.234415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:56.234921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:42:56.235300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.236197 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.238048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:42:56.238893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.239601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:42:56.239735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:42:56.240572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:42:56.241437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:42:56.241437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:00.760906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:00.761748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:00.762043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:00.763450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:00.797049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:00.797675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.798727 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:00.799623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:00.800328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:00.801130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:00.801969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:00.803047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:00.803047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:00.803047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:00.803047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:00.803047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.803047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:00.934735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:00.935510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:00.935685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:00.937212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:00.937586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:00.937835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.944972 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:00.946153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:00.946738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:00.947243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.947243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:00.947820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:00.948535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:00.949320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:00.949320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:00.949320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:01.094250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:01.095067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:01.095281 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:01.095471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:01.096528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:01.096917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.097965 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:01.099305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.101023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:01.101509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.101906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.102787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:01.103456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.104090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.104090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:01.104090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.104090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:01.185702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:01.186488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:01.186987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:01.186987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:01.187574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:01.188054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.188955 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:01.190251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.191089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:01.191815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.192524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:01.193329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:01.193838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.194392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:01.233104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:01.233540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:01.233705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:01.233953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:01.234489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:01.234840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:01.235122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.235826 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.237294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:01.237719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.238119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:01.238256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:01.238757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:01.239312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:01.239312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:05.761635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:05.762358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:05.762358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:05.763365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:05.797750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:05.797750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.798483 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:05.799380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:05.800473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:05.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:05.802046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:05.803219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:05.803219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:05.803219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:05.803219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:05.803219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.803219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:05.935576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:05.936197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:05.936338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:05.937077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:05.937282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:05.937412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.938025 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:05.938935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:05.939629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:05.940361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:05.941181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:05.942254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:05.942254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:05.942254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:05.942254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:05.942254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:05.942254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:06.095105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:06.096017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:06.096148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:06.096268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:06.097202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:06.097565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.098494 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:06.099497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.099982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.100333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:06.100695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.101155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.101155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:06.101155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.101155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.101597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:06.102112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.102651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.102651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:06.102651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.102651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:06.186455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:06.187124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:06.187276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:06.187452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:06.188136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:06.188455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.191702 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:06.193434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.194192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:06.194730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.195256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.195652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:06.195652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.195652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.196147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:06.196631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.197136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.197136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:06.197136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.197136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:06.233592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:06.234126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:06.234285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:06.234542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:06.235474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:06.235867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:06.236120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.236887 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.238436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:06.239155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.239726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:06.239978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:06.241382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:06.242334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:06.242334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:10.762462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:10.763292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:10.763572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:10.764615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:10.800837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:10.801207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.801994 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:10.802844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:10.803476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:10.804177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:10.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:10.806654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:10.806654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:10.806654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:10.806654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:10.806654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.806654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:10.936560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:10.937374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:10.937584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:10.939133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:10.939576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:10.939942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.941018 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:10.942922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:10.943649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:10.944269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.944269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:10.945255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:10.945786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:10.946427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:10.946427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:10.946427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:11.096235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:11.096995 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:11.097182 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:11.097326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:11.098100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:11.098592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.099824 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:11.100882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.101916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:11.102703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.103390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:11.104035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.104908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:11.187199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:11.187654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:11.187809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:11.188045 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:11.188636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:11.189798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.189963 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:11.191315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:11.192791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.193232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.193638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:11.193638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.193638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.194722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:11.195408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.196038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.196038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:11.196038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.196038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:11.234273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:11.234842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:11.234970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:11.235267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:11.235946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:11.236368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:11.236553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.237354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.238358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:11.238928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.239538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:11.239717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:11.240463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:11.241299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:11.241299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:15.763660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:15.764551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:15.764660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:15.766227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:15.802821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:15.803282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.804160 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:15.804945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:15.805764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:15.806494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:15.807398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:15.808541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:15.808541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:15.808541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:15.808541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:15.808541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.808541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:15.937818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:15.938577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:15.938790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:15.940351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:15.940803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:15.941103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.942165 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:15.943983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:15.944625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:15.945242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.945242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:15.945803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:15.946496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:15.947068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:15.947068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:15.947068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:16.097126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:16.097378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:16.097442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:16.097552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:16.098316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:16.098719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.099676 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:16.100513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.101342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:16.102050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.102824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:16.103453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.103962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:16.187845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:16.188470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:16.188617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:16.188791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:16.189478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:16.189888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.190767 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:16.192186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.192973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:16.193554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.194177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.194780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:16.194780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.194780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.195531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:16.196404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.197279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.197279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:16.197279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.197279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:16.234814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:16.235618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:16.235618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:16.235618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:16.235618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:16.236209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:16.236209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.236529 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.237352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:16.237985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:16.238593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:16.239360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:16.240130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:16.240130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:16.240130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:20.764400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:20.765516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:20.765516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:20.766871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:20.806557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:20.806990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.808855 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:20.809715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:20.810386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:20.810988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:20.811741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:20.812725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:20.812725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:20.812725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:20.812725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:20.812725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.812725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:20.938784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:20.939389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:20.939514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:20.940823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:20.941284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:20.941542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.942576 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:20.944379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:20.945134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:20.945779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.945779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:20.946159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:20.947047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:20.947600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:20.947600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:20.947600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:21.097797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:21.098315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:21.098409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:21.098480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:21.099187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:21.099433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.100156 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:21.101260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.102210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:21.102994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.103732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:21.104433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.105370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:21.188545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:21.189576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:21.189681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:21.189829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:21.190833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:21.190833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.191468 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:21.192311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.192761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:21.192974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.193336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.193693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:21.193693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.193693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.193779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:21.194232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.194718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.194718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:21.194718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.194718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:21.235715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:21.236277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:21.236441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:21.236875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:21.237187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:21.237585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:21.237839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.239168 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.241097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:21.241661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.241976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:21.243209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:21.243684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:21.244183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:21.244183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:25.765085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:25.766202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:25.766202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:25.767258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:25.805240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:25.805671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.806741 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:25.807883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:25.808787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:25.809752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:25.810815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:25.812105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:25.812105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:25.812105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:25.812105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:25.812105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.812105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:25.939571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:25.940348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:25.940557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:25.941854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:25.942133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:25.942280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.943168 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:25.944834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:25.945535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:25.946230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:25.947075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:25.947887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:25.947887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:25.947887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:25.947887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:25.947887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:25.947887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:26.098985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:26.099814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:26.099960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:26.100147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:26.100845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:26.101452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.102154 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:26.103325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.103869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.104638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:26.105099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.105524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.105524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:26.105524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.105524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.106032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:26.106614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.107281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.107281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:26.107281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.107281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:26.189483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:26.190022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:26.190190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:26.190367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:26.191381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:26.191812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.192931 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:26.195214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.195897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:26.196748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.197363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.197938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:26.197938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.197938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.198949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:26.199783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.200749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.200749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:26.200749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.200749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:26.236535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:26.237184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:26.237269 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:26.237498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:26.238452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:26.238820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:26.239096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.239988 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.241707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:26.242367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.243108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:26.243422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:26.244186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:26.244998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:26.244998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:30.766215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:30.766987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:30.767237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:30.768682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:30.806188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:30.806706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.807703 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:30.808615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:30.809213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:30.809814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:30.810527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:30.811461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:30.811461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:30.811461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:30.811461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:30.811461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.811461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:30.940562 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:30.941352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:30.941559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:30.943205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:30.943723 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:30.944115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.945305 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:30.947042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:30.947748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:30.948422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.948422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:30.948785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:30.949571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:30.950484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:30.950484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:30.950484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:31.100072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:31.100626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:31.100697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:31.100767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:31.101209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:31.101451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.102433 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:31.103769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.104300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.105421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:31.105816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.106281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.106281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:31.106281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.106281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.107371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:31.107878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.108539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.108539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:31.108539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.108539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:31.190163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:31.190910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:31.190994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:31.191216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:31.191873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:31.192351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.193202 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:31.194637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.195116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:31.196121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.196495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.196887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:31.196887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.196887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.198079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:31.198553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.199071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.199071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:31.199071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.199071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:31.237169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:31.237775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:31.237855 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:31.238116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:31.238703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:31.238994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:31.239197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.239788 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.240967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:31.241571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:31.242103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:31.242849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:31.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:31.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:31.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:35.767331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:35.768251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:35.768251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:35.769133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:35.771548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:43:35.807952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:35.808466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.809533 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:35.810468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:35.811102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:35.811671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:35.812327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:35.813198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:35.813198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:35.813198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:35.813198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:35.813198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.813198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:35.941633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:35.942607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:35.942845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:35.944375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:35.944698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:35.944979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.946123 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:35.947833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:35.948457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:35.949070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.949070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:35.949600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:35.950531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:35.951416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:35.951416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:35.951416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:36.100841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:36.101799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:36.101799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:36.102143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:36.103024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:36.103534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.104365 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:36.105800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.106583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.106973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:36.107616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.108252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:36.108947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.109830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:36.191257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:36.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:36.191772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:36.191974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:36.193178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:36.193665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.194799 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:36.196393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.197118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:36.197906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.198704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.199448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:36.199448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.199448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.199630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:36.200400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.201259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.201259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:36.201259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.201259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:36.237776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:36.238715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:36.238811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:36.239081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:36.239782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:36.240381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:36.240491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.241486 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.242680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:36.243609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:36.244347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:36.245182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:36.246118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:36.246118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:36.246118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:40.768210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:40.769166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:40.769166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:40.770123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:40.806508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:40.807093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.808241 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:40.809135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:40.809729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:40.810365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:40.810986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:40.811684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:40.811684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:40.811684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:40.811684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:40.811684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.811684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:40.942950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:40.944873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:40.944873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:40.945256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:40.945720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:40.946313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.947049 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:40.948304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:40.949087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:40.949796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:40.950630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:40.951698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:40.951698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:40.951698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:40.951698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:40.951698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:40.951698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:41.101805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:41.102302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:41.102454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:41.102611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:41.103141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:41.103380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.103950 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:41.105768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.106460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.107187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:41.107585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.107974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.107974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:41.107974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.107974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.108897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:41.109482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:41.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:41.192198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:41.194143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:41.194143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:41.194336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:41.195316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:41.195710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.197068 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:41.197866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.199033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:41.199674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:41.200148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:41.200568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.201197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:41.238356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:41.238896 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:41.239098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:41.239365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:41.240039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:41.240483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:41.240754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.241713 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.243272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:41.243940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.244528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:41.244871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:41.245737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:41.246461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:41.246461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:45.769070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:45.770141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:45.770141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:45.771097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:45.808628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:45.809132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.810284 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:45.811460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:45.812197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:45.812984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:45.813670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:45.814208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:45.814208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:45.814208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:45.814208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:45.814208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.814208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:45.943831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:45.945510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:45.945510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:45.946075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:45.946388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:45.946569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.947570 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:45.949161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:45.949599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:45.949962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.949962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:45.950991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:45.952738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:45.953781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:45.953781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:45.953781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:46.102612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:46.103682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:46.103682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:46.103682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:46.104483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:46.104846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.105575 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:46.106959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.107871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.108289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:46.108929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.109594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.109594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:46.109594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.109594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.110513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:46.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.111309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.111309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:46.111309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.111309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:46.194228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:46.194860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:46.195021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:46.195225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:46.196327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:46.196778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.197772 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:46.199493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.200266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:46.200863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.201614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.202304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:46.202304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.202304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.202641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:46.203510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.204413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.204413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:46.204413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.204413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:46.238948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:46.239354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:46.239485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:46.239739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:46.240345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:46.240719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:46.240968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.241832 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.243530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:46.243950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.244351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:46.245115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:46.245659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:46.246350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:46.246350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:50.769755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:50.770577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:50.770577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:50.772132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:50.815755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:50.816075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.817071 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:50.818149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:50.819548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:50.820064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:50.820819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:50.833432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:50.833432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:50.833432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:50.833432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:50.833432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.833432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:50.944867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:50.945285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:50.945473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:50.947040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:50.947489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:50.947914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.948823 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:50.950440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:50.951083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:50.951646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:50.952328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:50.953136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:50.953136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:50.953136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:50.953136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:50.953136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:50.953136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:51.103443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:51.104149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:51.104305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:51.104472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:51.105158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:51.105418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.106246 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:51.107205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.108052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:51.108778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.109408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:51.110084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.110889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:51.195234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:51.195551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:51.195696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:51.195873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:51.196978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:51.197535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.198530 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:51.200322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.201077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:51.201772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.202585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.203273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:51.203273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.203273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.203475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:51.204300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.205209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.205209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:51.205209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.205209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:51.239564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:51.239848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:51.239955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:51.240342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:51.241223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:51.241656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:51.241901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.243053 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.244447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:51.245126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.245714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:51.246211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:51.247117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:51.248068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:51.248068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:55.770514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:55.771331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:55.771413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:55.772413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:55.816716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:55.817427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.818324 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:55.819467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:55.819894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:55.820299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.820299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:55.820682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:55.821229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:55.821812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:55.821812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.821812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:55.962906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:55.963712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:55.963712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:55.964770 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:55.965047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:55.965196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.965852 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:55.966725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:55.967014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:55.967281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.967281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:55.967489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:55.967824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:55.968213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:55.968213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:55.968213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:56.104371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:56.105172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:56.105270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:56.105670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:43:56.106735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:56.107162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.108019 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:56.109048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.110097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:56.110920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.111672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:56.112396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.113363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:56.196026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:56.196533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:56.196719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:56.196918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:56.197709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:56.198336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.199336 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:43:56.200720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.201457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:56.202175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.202568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.202957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:56.202957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.202957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.203361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:56.216820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.217807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.217807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:56.217807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.217807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:43:56.240215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:43:56.240882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:43:56.241137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:43:56.241359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:43:56.242283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:56.242667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:43:56.242863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.243562 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.245172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:43:56.245792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.246456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:43:56.246663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:43:56.247494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:43:56.248293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:43:56.248293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:00.771363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:00.772800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:00.772800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:00.773554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:00.818954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:00.819453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:00.820673 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:00.821660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:00.822522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:00.823211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:00.823933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:00.824671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:00.824671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:00.824671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:00.824671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:00.824671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:00.824671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:00.963900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:01.027242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:01.027242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:01.031654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:01.031887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:01.032071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.033321 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.034307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.034587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:01.034864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.034864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:01.035492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.036077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.036687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:01.036687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.036687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:01.105624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:01.106364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:01.106532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:01.106660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:01.107760 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:01.107943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.108939 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:01.110109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.110969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:01.111712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.112397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:01.113171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.114178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:01.196736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:01.197242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:01.197378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:01.197654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:01.198545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:01.199030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.199928 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:01.201515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.202588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.203494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.204354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.205340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:01.205340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.205340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.205872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:01.206692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.207265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.207265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:01.207265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.207265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:01.240810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:01.241719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:01.241886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:01.242256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:01.243094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:01.243511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:01.243766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.244648 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.245920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:01.246579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:01.247232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:01.248038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:01.248877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:01.248877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:01.248877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:05.771977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:05.772924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:05.772924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:05.773949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:05.807411 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:05.807915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.808890 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:05.809701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:05.810183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:05.810573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:05.811074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.811631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:05.964710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:05.969627 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:05.970235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:05.970235 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:05.970235 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.970235 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:05.970540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:05.971141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:05.971690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.971690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:05.972364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:05.973161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:05.976899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:05.976899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:05.976899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:06.106328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:06.106955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:06.107076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:06.107200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:06.107704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:06.107937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.108569 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:06.109506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.110119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.110547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:06.110944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.111374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.111374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:06.111374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.111374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.111700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:06.112293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.112872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.112872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:06.112872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.112872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:06.197670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:06.198654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:06.198654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:06.198979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:06.199714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:06.200045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.200879 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:06.202149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.202643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:06.203439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.203864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.204241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:06.204241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.204241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.205121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:06.205673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.206238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.206238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:06.206238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.206238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:06.241528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:06.242352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:06.242532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:06.242833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:06.243429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:06.243810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:06.244122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.244891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.247434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:06.248230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.248886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:06.249310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:06.250303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:06.251408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:06.251408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:10.776485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:10.777651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:10.777651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:10.778787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:10.820261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:10.820770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.821413 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:10.822513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:10.823225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:10.823863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:10.824717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:10.825342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:10.825342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:10.825342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:10.825342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:10.825342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.825342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:10.966095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:10.967486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:10.968295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:10.973220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:10.973927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:10.974090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.974771 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:10.975713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:10.976400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:10.976980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:10.977740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:10.978643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:10.978643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:10.978643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:10.978643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:10.978643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:10.978643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:11.107222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:11.107682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:11.107824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:11.108095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:11.109245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:11.109722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.110704 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:11.111728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.112755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:11.113781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.114595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:11.115654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.116958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:11.198718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:11.199295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:11.199387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:11.199499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:11.200481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:11.201114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.202206 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:11.202936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.203696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:11.204432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:11.205164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:11.205680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.206476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:11.242347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:11.242756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:11.242880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:11.242989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:11.243481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:11.243769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:11.243892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.244536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.246222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:11.247098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.247782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:11.249262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:11.249901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:11.250596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:11.250596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:15.777735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:15.779882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:15.780139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:15.782847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:15.818622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:15.819068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.820555 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:15.822879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:15.823639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:15.824378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.824378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:15.824504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:15.825346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:15.826478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:15.826478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.826478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:15.967052 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:15.968179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:15.968179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:15.969353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:15.969671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:15.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.970873 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:15.972324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:15.972903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:15.973459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:15.974232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:15.975071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:15.975071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:15.975071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:15.975071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:15.975071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:15.975071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:16.107922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:16.115131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:16.115470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:16.115619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:16.116541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:16.116931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.117953 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:16.119450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.120317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.120725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:16.121365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.121968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.121968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:16.121968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.121968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.122378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:16.123169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.123778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.123778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:16.123778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.123778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:16.199659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:16.200277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:16.200463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:16.200708 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:16.201511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:16.201972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.202897 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:16.204450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.205212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:16.206024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.206655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.207309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:16.207309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.207309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.207913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:16.208706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.209555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.209555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:16.209555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.209555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:16.242813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:16.243184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:16.243387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:16.243620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:16.244363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:16.244644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:16.244840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.246319 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.247028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:16.247718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.248312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:16.248683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:16.249475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:16.250283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:16.250283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:20.778782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:20.780087 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:20.780087 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:20.781075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:20.826242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:20.827245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Traceback (most recent call last):
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.828171 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:20.829339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:20.830083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8b241f6-ebb4-485e-8148-1a8fdf1935c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:20.830561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:20.831027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:20.831561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:20.831561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:20.831561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:20.831561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:20.831561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.831561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:20.968125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:20.969160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:20.969294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:20.970619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:20.970800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:20.970982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.971636 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:20.972631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:20.973561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:20.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:20.974856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:20.975566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:20.975566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:20.975566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:20.975566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:20.975566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:20.975566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:21.115173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:21.115573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:21.115693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:21.115804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:21.116665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:21.116925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.117641 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:21.118645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.119546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:21.120335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.121072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:21.121941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.122972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:21.200357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:21.200676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:21.200772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:21.200958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:21.202094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:21.203239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.203740 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:21.205186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.205956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:21.206538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.207163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:21.207875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:21.208642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.209722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:21.243338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:21.245313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:21.245433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:21.245696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:21.246435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:21.247142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:21.247594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.248257 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.249694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:21.250435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.251065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:21.251243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:21.252115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:21.252933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:21.252933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:25.779722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:25.780912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:25.780912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:25.782037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:25.820267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:25.820787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.821870 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:25.822764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:25.823472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:25.824168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:25.824979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:25.826126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:25.826126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:25.826126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:25.826126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:25.826126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.826126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:25.969241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:25.969524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:25.969735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:25.970758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:25.971016 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:25.971279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.972186 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:25.973798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:25.974557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:25.975127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.975127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:25.975490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:25.976865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:25.977899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:25.977899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:25.977899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:26.115925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:26.116407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:26.116553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:26.116691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:26.117239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:26.117486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.118131 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:26.119462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.120395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.120837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:26.121448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.122079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.122079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:26.122079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.122079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.122579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:26.123430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.124407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.124407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:26.124407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.124407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:26.201075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:26.201760 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:26.201888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:26.202040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:26.202583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:26.202929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.203607 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:26.204369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.205125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:26.205452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.206047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:26.206803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:26.207531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.208565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:26.245329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:26.245825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:26.245974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:26.246287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:26.247063 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:26.247465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:26.247714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.249708 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.251771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:26.252467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.252990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:26.253486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:26.254036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:26.254537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:26.254537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:30.780540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:30.781375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:30.781375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:30.782198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:30.819592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:30.819934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.820809 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:30.821864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:30.822830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:30.823699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:30.824640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:30.825423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:30.825423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:30.825423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:30.825423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:30.825423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.825423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:30.970142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:30.970682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:30.970797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:30.972474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:30.972897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:30.973190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.974183 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:30.977538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:30.978211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:30.978810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:30.979668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:30.980251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:30.980251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:30.980251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:30.980251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:30.980251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:30.980251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:31.116713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:31.117031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:31.117205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:31.117358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:31.118169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:31.118470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.119229 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:31.120915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.121773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.122283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:31.122900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.123575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.123575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:31.123575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.123575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.123920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:31.124443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.125046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.125046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:31.125046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.125046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:31.201803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:31.202384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:31.202531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:31.202702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:31.203307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:31.203716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.204561 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:31.205739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.206440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:31.207273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.207907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.208511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:31.208511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.208511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.208658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:31.209448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.210588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.210588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:31.210588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.210588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:31.246889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:31.247591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:31.247778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:31.248059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:31.248844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:31.249306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:31.249947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.250772 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.252585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:31.253332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.253943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:31.254158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:31.254838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:31.255372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:31.255372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:35.782545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:35.784153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:35.784153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:35.785155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:35.788495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:44:35.829171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:35.829171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.830474 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:35.831253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:35.831786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:35.832360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.832360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:35.832579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:35.833341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:35.834434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:35.834434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.834434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:35.971068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:35.971581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:35.972673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:35.972988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:35.973452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:35.973748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.974732 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:35.975943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:35.976666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:35.977343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:35.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:35.979032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:35.979032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:35.979032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:35.979032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:35.979032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:35.979032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:36.117493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:36.118475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:36.119112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:36.119306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:36.120136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:36.120399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.121190 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:36.122562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.123478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.124027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:36.124685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:36.125323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.126082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.126923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:36.202673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:36.203276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:36.203415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:36.203589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:36.204522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:36.204992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.205977 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:36.207772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.208666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:36.209056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.209590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:36.210198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:36.210930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.211805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:36.305283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:36.305283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:36.305427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:36.305844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:36.306088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:36.306310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:36.306447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.306975 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.308076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:36.308515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.308878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:36.309133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:36.309614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:36.310207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:36.310207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:40.782751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:40.783547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:40.783547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:40.784435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:40.818794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:40.819293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.820437 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:40.821224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:40.821604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:40.821988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:40.822462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:40.822989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:40.822989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:40.822989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:40.822989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:40.822989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.822989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:40.971874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:40.972620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:40.972857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:40.974351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:40.974752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:40.975013 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.975971 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:40.977378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:40.977825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:40.978228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.978228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:40.978704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:40.979216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:40.979749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:40.979749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:40.979749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:41.118568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:41.119398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:41.119578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:41.119915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:41.120992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:41.121371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.122409 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:41.124033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.125034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.125586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:41.126120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.126671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.126671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:41.126671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.126671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.127032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:41.127520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.128159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.128159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:41.128159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.128159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:41.203655 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:41.204388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:41.204581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:41.204835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:41.205851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:41.206332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.207386 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:41.209190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.210296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:41.211292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.212088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.212646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:41.212646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.212646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.213099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:41.213602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.214154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.214154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:41.214154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.214154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:41.305823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:41.356578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:41.356578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:41.356802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:41.357583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:41.357848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:41.358178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.358469 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.359151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:41.359491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.359810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:41.360138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:41.360609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:41.361100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:41.361100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:45.783718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:45.784915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:45.784915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:45.785985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73c9a4f2-bfbd-47a7-87a4-0369593088de None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:45.825484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:45.825908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73c9a4f2-bfbd-47a7-87a4-0369593088de None None] Traceback (most recent call last):
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.826898 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73c9a4f2-bfbd-47a7-87a4-0369593088de None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:45.828256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:45.828917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:45.829537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.829537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73c9a4f2-bfbd-47a7-87a4-0369593088de None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:45.829884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:45.830512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:45.831358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:45.831358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.831358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:45.972818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:45.973562 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:45.973745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:45.975049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:45.975436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:45.975695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.976769 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:45.978594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:45.979380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:45.980046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.980046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:45.980610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:45.981489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:45.982471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:45.982471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:45.982471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:46.119316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:46.120085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:46.120085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:46.120429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:46.120902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:46.121316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.122181 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:46.123343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.124350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:46.125231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.126048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:46.126666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:46.204486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:46.205155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:46.205325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:46.205508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:46.206521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:46.207073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.207980 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:46.209626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.210606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:46.211448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.212120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:46.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:46.213738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.214662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:46.307196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:46.307838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:46.307977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:46.308253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:46.308962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:46.309442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:46.309683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.310849 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.312552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:46.313593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:46.314216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:46.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:46.315997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:46.315997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:46.315997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:50.784352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:50.785128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:50.785128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:50.786177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:50.824739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:50.825264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Traceback (most recent call last):
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.826512 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:50.827240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:50.827621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:50.827952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:50.828402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:50.828988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:50.828988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:50.828988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:50.828988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:50.828988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.828988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:50.973715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:50.974357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:50.974535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:50.975691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:50.976158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:50.976460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.977624 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:50.979098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:50.979738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:50.980280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.980280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:50.980540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:50.981316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:50.981991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:50.981991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:50.981991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:51.120110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:51.120720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:51.120833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:51.121044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:51.121892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:51.122322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.123306 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:51.124464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.125333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.126137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:51.126753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.127189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.127189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:51.127189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.127189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.127700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:51.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.128818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.128818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:51.128818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.128818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:51.205133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:51.206251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:51.206532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:51.206751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:51.207235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:51.207485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.208031 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:51.208847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.209888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:51.210625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:51.211165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:51.211534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.212153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:51.307926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:51.308460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:51.308567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:51.308757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:51.309304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:51.309744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:51.309983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.310877 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.311837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:51.312372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.312734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:51.312952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:51.313529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:51.314113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:51.314113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:55.785157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:55.785841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:55.785961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:55.787270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:55.823742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:55.824088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.825029 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:55.826316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:55.826993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:55.827824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:55.828712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:55.829767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:55.829767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:55.829767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:55.829767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:55.829767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.829767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:55.974578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:55.975258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:55.975412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:55.976911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:55.977230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:55.977365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.978131 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:55.979201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:55.979615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:55.980193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.980193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:55.980317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:55.980776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:55.981329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:55.981329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:55.981329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:56.120908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:56.121606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:56.121734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:56.122111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:44:56.123101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:56.123534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.124648 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:56.126186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.127465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.128238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:56.129059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.130066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.130066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:56.130066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.130066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.130368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:56.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.131532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.131532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:56.131532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.131532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:56.206405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:56.207049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:56.207207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:56.207398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:56.208326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:56.208794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.209953 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:44:56.211280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.212049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:56.212863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.213474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:56.214246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:56.214951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.215775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:44:56.309226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:44:56.309624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:44:56.309791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:44:56.310099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:44:56.310982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:56.311518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:44:56.311739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.312700 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.314388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:44:56.314779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.315156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:44:56.316183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:44:56.316681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:44:56.317188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:44:56.317188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:00.786244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:00.787656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:00.787781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:00.789122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:00.828053 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:00.828609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Traceback (most recent call last):
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.829914 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:00.831172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:00.831866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:00.832378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:00.832829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:00.833579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:00.833579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:00.833579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:00.833579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:00.833579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.833579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:00.975625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:00.976284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:00.976442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:00.977626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:00.978082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:00.978330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.979434 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:00.981453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:00.982082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:00.982734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.982734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:00.983365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:00.984138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:00.985039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:00.985039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:00.985039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:01.121826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:01.122599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:01.122913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:01.123027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:01.124246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:01.124883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.125872 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:01.127372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.128229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.128640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:01.129270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.129883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.129883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:01.129883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.129883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.130234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:01.131088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.131843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.131843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:01.131843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.131843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:01.207359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:01.207897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:01.207897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:01.208196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:01.209207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:01.209711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.210768 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:01.211958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.212354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:01.213334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.213754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.214131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:01.214131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.214131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.214989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:01.215693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.216224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.216224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:01.216224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.216224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:01.310250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:01.311039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:01.311229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:01.311525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:01.312118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:01.312530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:01.312734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.313460 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.314990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:01.315725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.316415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:01.316676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:01.317591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:01.318170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:01.318170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:05.787848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:05.788893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:05.788893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:05.789568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:05.827374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:05.827737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.828762 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:05.829829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:05.830556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:05.831349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:05.832190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:05.833166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:05.833166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:05.833166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:05.833166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:05.833166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.833166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:05.976526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:05.977200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:05.977484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:05.978675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:05.979106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:05.979402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.980422 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:05.981916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:05.982559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:05.983140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.983140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:05.983571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:05.984566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:05.985521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:05.985521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:05.985521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:06.122653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:06.123342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:06.123499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:06.123659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:06.124523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:06.124814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.125562 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:06.126984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.127867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.128307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:06.128893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.129539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.129539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:06.129539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.129539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.130552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:06.131580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.132547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.132547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:06.132547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.132547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:06.207878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:06.208598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:06.208736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:06.209210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:06.209865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:06.210383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.211497 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:06.212891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.213356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:06.214472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.214852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.215247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:06.215247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.215247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.215542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:06.216034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.216557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.216557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:06.216557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.216557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:06.310908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:06.311404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:06.311500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:06.311711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:06.312503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:06.312911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:06.313170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.314250 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.315615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:06.316388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.317030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:06.317220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:06.318155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:06.318998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:06.318998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:10.788738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:10.789585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:10.789783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:10.791405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:10.831822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:10.832449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.833949 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:10.835270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:10.835941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:10.836588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:10.837513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:10.838437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:10.838437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:10.838437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:10.838437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:10.838437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.838437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:10.977421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:10.978185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:10.978417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:10.979852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:10.981151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:10.981151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Traceback (most recent call last):
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.981906 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:10.983106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:10.983512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:10.983868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.983868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-52794357-f143-4a1e-bfb5-dc5b6ef86a32 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:10.984327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:10.984919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:10.985511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:10.985511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:10.985511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:11.123534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:11.124367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:11.124629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:11.124629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:11.125455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:11.126084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.126974 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:11.128401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.128971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.129706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:11.130119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.130498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.130498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:11.130498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.130498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.131368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:11.131886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.132527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.132527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:11.132527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.132527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:11.208631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:11.209394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:11.209621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:11.210199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:11.210705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:11.210968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.211721 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:11.213493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.214729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:11.215644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.216755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.217454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:11.217454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.217454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.217988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:11.218511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.219396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.219396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:11.219396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.219396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:11.311759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:11.312306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:11.312432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:11.313145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:11.313204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:11.313426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:11.313584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.314148 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.315688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:11.316198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.316612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:11.317047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:11.317576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:11.318191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:11.318191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:15.789820 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:15.790842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:15.790842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:15.792338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:15.829858 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:15.830563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Traceback (most recent call last):
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.831765 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:15.832966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:15.833652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:15.834365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:15.835250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:15.836292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:15.836292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:15.836292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:15.836292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:15.836292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.836292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:15.978274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:15.979207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:15.979424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:15.980621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:15.980946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:15.981277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.982211 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:15.983377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:15.983969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:15.984585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:15.985287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:15.986153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:15.986153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:15.986153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:15.986153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:15.986153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:15.986153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:16.124435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:16.125208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:16.125399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:16.125570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:16.127185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:16.127567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.128513 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:16.129508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.130419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:16.131170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.131530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:16.131890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.132409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:16.209291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:16.209678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:16.209797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:16.209972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:16.211021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:16.211556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.212562 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:16.213925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.214876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:16.215499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.216101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.216717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:16.216717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.216717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.217447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:16.218272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.219318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.219318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:16.219318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.219318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:16.312598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:16.313254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:16.313404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:16.313674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:16.314515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:16.314813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:16.314969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.315622 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.317235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:16.317641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.318017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:16.318939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:16.319444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:16.319926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:16.319926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:20.790694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:20.791478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:20.791609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:20.792546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:20.836333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:20.836869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Traceback (most recent call last):
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.838456 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:20.839860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:20.840777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-08a92706-51c0-4b60-b70f-094148aa7697 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:20.841836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:20.842810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:20.844161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:20.844161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:20.844161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:20.844161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:20.844161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.844161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:20.979208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:20.979985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:20.980149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:20.981445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:20.981770 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:20.981957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.982699 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:20.983948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:20.984483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:20.984815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.984815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:20.985146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:20.985654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:20.986117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:20.986117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:20.986117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:21.125386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:21.126144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:21.126494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:21.126565 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:21.127570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:21.127957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.129149 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:21.130696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.132381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:21.135947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.137469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:21.138216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.138921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:21.210121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:21.211137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:21.211137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:21.211137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:21.211853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:21.212431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.213706 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:21.214982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.215491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:21.216516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.216904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.217390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:21.217390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.217390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.219096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:21.219707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.220135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.220135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:21.220135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.220135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:21.313335 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:21.314029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:21.314207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:21.314497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:21.315148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:21.315377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:21.315562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.316278 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.318116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:21.318513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.318892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:21.319774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:21.320317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:21.320828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:21.320828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:25.791774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:25.792930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:25.792930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:25.794450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:25.827953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:25.828340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] Traceback (most recent call last):
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.829276 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:25.830490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:25.831318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-8e4b1028-a79f-44c7-9880-01731e10c270 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:25.832100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:25.832992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:25.833970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:25.833970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:25.833970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:25.833970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:25.833970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.833970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:25.980336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:25.980978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:25.981188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:25.982706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:25.983165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:25.983455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.984507 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:25.985595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:25.986342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:25.986917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.986917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:25.988630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:25.990096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:25.991037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:25.991037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:25.991037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:26.126279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:26.127033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:26.127210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:26.127369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:26.128191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:26.128610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.129616 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:26.131361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.132801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:26.133636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.134352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:26.135137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.136122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:26.211060 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:26.211940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:26.212180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:26.212468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:26.213104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:26.213489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.214313 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:26.215319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.215993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:26.216299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.216663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.217053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:26.217053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.217053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.217377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:26.217899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.218471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.218471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:26.218471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.218471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:26.314345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:26.315107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:26.315343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:26.315565 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:26.316650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:26.317206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:26.317478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.318499 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.320253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:26.320882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.321542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:26.321795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:26.322591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:26.323415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:26.323415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:30.792597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:30.793590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:30.793590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:30.794627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:30.830628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:30.831288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.832288 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:30.833446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:30.834228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:30.834949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:30.835799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:30.836774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:30.836774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:30.836774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:30.836774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:30.836774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.836774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:30.981109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:30.981830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:30.982110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:30.983697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:30.984197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:30.984568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.985676 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:30.986887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:30.987433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:30.987834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.987834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:30.987952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:30.988487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:30.989190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:30.989190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:30.989190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:31.126963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:31.127358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:31.127509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:31.127662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:31.128625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:31.129023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.130476 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:31.131929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.132751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.133981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:31.134718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.135457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.135457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:31.135457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.135457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.136064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:31.136833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.137532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.137532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:31.137532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.137532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:31.211882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:31.212213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:31.212378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:31.212559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:31.213580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:31.214085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.215120 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:31.216628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.217137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:31.218079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.218446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.218836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:31.218836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.218836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.220158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:31.220701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.221282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.221282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:31.221282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.221282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:31.315284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:31.316150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:31.316289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:31.316563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:31.317384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:31.317810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:31.318085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.319090 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.320596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:31.321367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.322040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:31.322112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:31.322981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:31.323938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:31.323938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:35.793421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:35.793982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:35.793982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:35.795289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:35.818275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:45:35.836420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:35.836805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.838218 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:35.839425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:35.840195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:35.840937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:35.841752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:35.842677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:35.842677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:35.842677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:35.842677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:35.842677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.842677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:35.982025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:35.983034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:35.983340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:35.985257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:35.985257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:35.985623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.986100 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:35.987500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:35.988023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:35.988379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.988379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:35.988743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:35.989243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:35.989816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:35.989816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:35.989816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:36.127856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:36.128548 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:36.128749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:36.128859 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:36.129894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:36.130298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.131528 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:36.132309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.133168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:36.134066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.134699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:36.135375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.136245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:36.212724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:36.213273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:36.213367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:36.213496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:36.214241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:36.214727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.215587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:36.216488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.217202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:36.217450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.217847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:36.218779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:36.219280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.219905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:36.316402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:36.316968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:36.317180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:36.317577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:36.318177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:36.318561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:36.318803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.319710 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.322305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:36.323281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:36.323877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:36.324685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:36.325590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:36.325590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:36.325590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:40.794168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:40.795726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:40.795726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:40.797248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:40.839611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:40.839862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.840512 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:40.841722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:40.842517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:40.843231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:40.844101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:40.845110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:40.845110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:40.845110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:40.845110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:40.845110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.845110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:40.983126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:40.983738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:40.983924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:40.985081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:40.985275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:40.985435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.986195 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:40.987765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:40.988280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:40.988665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.988665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:40.989728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:40.990277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:40.990832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:40.990832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:40.990832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:41.128577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:41.129151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:41.129540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:41.129835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:41.130759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:41.131106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.131795 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:41.132682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.133793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:41.134475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.135069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:41.135651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.136740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:41.213564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:41.214079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:41.214220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:41.214379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:41.215263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:41.215714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.216607 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:41.217805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.218594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:41.219195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.219827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.220480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:41.220480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.220480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.220811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:41.221670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.222610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.222610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:41.222610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.222610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:41.316954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:41.317748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:41.317897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:41.318227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:41.319101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:41.319563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:41.319989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.320817 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.322486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:41.323162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.323732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:41.324372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:41.325217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:41.326062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:41.326062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:45.795041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:45.795925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:45.796190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:45.797547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:45.835203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:45.835490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.836472 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:45.837291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:45.838059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:45.838784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:45.839660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:45.840725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:45.840725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:45.840725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:45.840725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:45.840725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.840725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:45.983943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:45.984632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:45.984841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:45.985861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:45.986191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:45.986482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.987367 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:45.988987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:45.989599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:45.990209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.990209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:45.990581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:45.991444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:45.992400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:45.992400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:45.992400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:46.129425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:46.129802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:46.129883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:46.129990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:46.131464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:46.131782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.132745 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:46.133726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.134672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.134924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:46.135336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.135727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.135727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:46.135727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.135727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.135845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:46.136378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.137096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.137096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:46.137096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.137096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:46.214250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:46.214865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:46.215040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:46.215268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:46.215788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:46.216133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.216768 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:46.217627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.218287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:46.218624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.219110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:46.219595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:46.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.220907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:46.317940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:46.318604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:46.318838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:46.319135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:46.320164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:46.320624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:46.320910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.321897 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.323831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:46.324357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.324714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:46.324943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:46.325455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:46.325966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:46.325966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:50.795909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:50.796846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:50.797115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:50.798493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:50.844339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:50.844662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Traceback (most recent call last):
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.845690 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:50.846937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:50.847533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:50.848175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.848175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-420be5f8-638a-47c6-a46f-bcbea77d9ce1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:50.848465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:50.849197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:50.850153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:50.850153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.850153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:50.985495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:50.987225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:50.987225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:50.987225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:50.987225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:50.987225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.988668 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:50.989905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:50.990824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:50.991267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.991267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:50.991386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:50.992076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:50.992938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:50.992938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:50.992938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:51.130132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:51.130785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:51.131019 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:51.131260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:51.131980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:51.132362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.133436 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:51.134864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.135630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.136278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:51.136684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.137227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.137227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:51.137227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.137227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.138086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:51.138789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.139548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.139548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:51.139548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.139548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:51.215057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:51.215863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:51.216293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:51.216293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:51.217335 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:51.217823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.219039 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:51.220103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.221172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:51.221644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.222308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:51.223222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:51.224208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.225365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:51.318594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:51.319539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:51.319539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:51.319829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:51.320625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:51.321231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:51.321631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.322535 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.324193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:51.324836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.325615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:51.325718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:51.326510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:51.327337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:51.327337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:55.796709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:55.797571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:55.797773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:55.799483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:55.844169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:55.844499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Traceback (most recent call last):
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.845465 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:55.846408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:55.847054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-5b3e079f-9572-4056-b804-b53d81e751f9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:55.847716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:55.848511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:55.849473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:55.849473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:55.849473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:55.849473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:55.849473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.849473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:55.986159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:55.986626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:55.986795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:55.988471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:55.988951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:55.989346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.990623 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:55.992877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:55.994093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:55.994713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.994713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:55.994935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:55.995764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:55.996892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:55.996892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:55.996892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:56.131040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:56.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:56.131940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:56.132149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:45:56.133210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:56.133585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.134646 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:56.135978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.136788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.137424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:56.137986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.138662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.138662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:56.138662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.138662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.139166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:56.139953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.140920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.140920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:56.140920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.140920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:56.215889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:56.216286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:56.216378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:56.216449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:56.217493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:56.217941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.218817 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:45:56.220244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.220729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:56.222243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.222648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.223050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:56.223050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.223050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.224505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:56.225305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.225968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.225968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:56.225968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.225968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:45:56.319295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:45:56.319854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:45:56.319994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:45:56.320394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:45:56.321241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:56.321512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:45:56.321743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.323117 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.324502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:45:56.325157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:45:56.325800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:45:56.326806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:45:56.327941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:45:56.327941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:45:56.327941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:00.797765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:00.799074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:00.799074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:00.800105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:00.839347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:00.839555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.840310 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:00.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:00.842324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:00.843057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:00.844084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:00.845102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:00.845102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:00.845102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:00.845102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:00.845102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.845102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:00.986953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:00.987909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:00.988249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:00.989537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:00.989744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:00.989860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.990626 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:00.992163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:00.993108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:00.994301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:00.995373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:00.996624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:00.996624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:00.996624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:00.996624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:00.996624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:00.996624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:01.131844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:01.132636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:01.132809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:01.132977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:01.134080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:01.134339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.135152 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:01.136540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.137644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.138314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:01.139253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.139898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.139898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:01.139898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.139898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.140158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:01.140925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.141846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.141846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:01.141846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.141846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:01.216702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:01.217636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:01.217972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:01.218346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:01.219375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:01.219836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.220868 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:01.222519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.223210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.223971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:01.224560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.225199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.225199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:01.225199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.225199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.225814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:01.226512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.227031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.227031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:01.227031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.227031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:01.320283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:01.320661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:01.320781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:01.321099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:01.321975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:01.322494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:01.322750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.323695 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.324881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:01.325752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:01.326365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:01.327139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:01.327931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:01.327931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:01.327931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:05.798604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:05.800127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:05.800127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:05.800690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:05.838702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:05.839189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.840153 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:05.840944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:05.841391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:05.841778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:05.842226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:05.842739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:05.842739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:05.842739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:05.842739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:05.842739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.842739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:05.987936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:05.988751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:05.988968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:05.990384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:05.990781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:05.991082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.992107 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:05.993801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:05.995555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:05.996127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:05.996858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:05.997648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:05.997648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:05.997648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:05.997648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:05.997648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:05.997648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:06.132717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:06.133524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:06.133670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:06.133828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:06.134746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:06.135259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.136098 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:06.137604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.138069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.139015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:06.139294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.139560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.139560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:06.139560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.139560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.140718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:06.141109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.143082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.143082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:06.143082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.143082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:06.217536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:06.218252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:06.218381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:06.218540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:06.219257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:06.219580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.220332 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:06.221520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.222184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.223134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:06.223719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.224526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.224526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:06.224526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.224526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.224992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:06.225781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.226326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.226326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:06.226326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.226326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:06.321174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:06.321897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:06.322191 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:06.322499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:06.323102 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:06.323544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:06.323794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.324634 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.326367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:06.327126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.327816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:06.328277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:06.329303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:06.330398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:06.330398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:10.799230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:10.800146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:10.800146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:10.800648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:10.840150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:10.840583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.841518 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:10.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:10.843048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:10.843694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:10.844422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:10.845246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:10.845246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:10.845246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:10.845246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:10.845246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.845246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:10.988984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:10.991486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:10.991697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:10.993276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:10.993573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:10.993765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.994545 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:10.995605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:10.996284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:10.997023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:10.997945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:10.999142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:10.999142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:10.999142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:10.999142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:10.999142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:10.999142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:11.133740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:11.134369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:11.134511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:11.134668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:11.135587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:11.135938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.137641 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:11.140335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.141214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:11.142181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.142633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.143083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:11.143607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.144162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.144162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:11.144162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.144162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:11.218406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:11.218905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:11.219183 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:11.219417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:11.220198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:11.220465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.221239 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:11.222764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.223437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.224462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:11.224914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.225321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.225321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:11.225321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.225321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.226610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:11.227117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.227631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.227631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:11.227631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.227631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:11.321926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:11.322810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:11.322938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:11.323235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:11.324184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:11.324659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:11.324900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.325912 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.327476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:11.328318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:11.329072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:11.329921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:11.331207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:11.331207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:11.331207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:15.800079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:15.801073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:15.801280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:15.802245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:15.834247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.834891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:15.836698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:15.837835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:15.838530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:15.839936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:15.841066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:15.841066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:15.841066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:15.841066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:15.841066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.841066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:15.991784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:15.992609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:15.992838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:15.994293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:15.994704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:15.994960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.996069 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:15.998203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:15.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:15.999163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:15.999163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:16.000216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.001073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.001609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:16.001609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.001609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:16.134337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:16.135485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:16.135562 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:16.136491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:16.136867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.137816 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:16.138945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.140613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:16.141707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.142471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:16.143187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.144167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:16.218970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:16.219738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:16.219931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:16.220187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:16.221416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:16.221416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.222356 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:16.223796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.224592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.225366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:16.225953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.226699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.226699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:16.226699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.226699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.227225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:16.227869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.228401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.228401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:16.228401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.228401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:16.323186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:16.324170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:16.324170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:16.324866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:16.325366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:16.326311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:16.326589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.327577 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.328811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:16.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.329744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:16.330606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:16.331125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:16.331617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:16.331617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:20.800961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:20.802210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:20.802210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:20.803454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:20.840459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:20.840901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.841972 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:20.844226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:20.844926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:20.845642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:20.846506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:20.847450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:20.847450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:20.847450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:20.847450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:20.847450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.847450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:20.992530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:20.993339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:20.993547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:20.994700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:20.995038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:20.995311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.996305 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:20.997837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:20.998588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:20.999044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.999044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:20.999625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.000127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.000629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:21.000629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.000629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:21.136651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:21.137268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:21.137371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:21.137487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:21.138301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:21.138538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.139367 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:21.140405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.141116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.141568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:21.141905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.142301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:21.142705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.143269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:21.219788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:21.230863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:21.230863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:21.230863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:21.231360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:21.231623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.232283 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:21.233191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.233579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:21.233923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.234294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.234642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:21.234642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.234642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.234828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:21.235320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.235762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.235762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:21.235762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.235762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:21.323893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:21.324578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:21.324676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:21.324975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:21.325720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:21.326121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:21.326364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.327245 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.328879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:21.329593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:21.330384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:21.331265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:21.332208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:21.332208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:21.332208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:25.801783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:25.802962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:25.802962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:25.804142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:25.841856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:25.842231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.843236 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:25.844591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:25.845098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:25.845490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.845490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:25.845575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:25.846141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:25.846745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:25.846745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.846745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:25.993339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:25.993940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:25.994149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:25.995220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:25.995873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:25.996165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.997196 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:25.998669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:25.999563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:26.000410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:26.001422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.002698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:26.002698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.002698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.002698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:26.002698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.002698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:26.137245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:26.137890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:26.138072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:26.138235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:26.139206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:26.139564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.140612 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:26.142874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.143452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.144438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:26.144845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.145255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.145255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:26.145255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.145255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.146112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:26.146603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.147189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.147189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:26.147189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.147189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:26.230570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:26.230785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:26.230907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:26.230976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:26.231941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:26.232413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.233335 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:26.234186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.234949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:26.235806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:26.236493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.237191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.238174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:26.324307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:26.324599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:26.324599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:26.324769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:26.325731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:26.326776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:26.326776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.328392 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.329284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:26.330045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:26.330451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:26.330961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:26.331566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:26.331566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:26.331566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:30.802570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:30.803618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:30.803618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:30.804751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:30.849735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:30.850230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.851371 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:30.852699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:30.853413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:30.854096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:30.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:30.855909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:30.855909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:30.855909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:30.855909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:30.855909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.855909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:30.994200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:30.995028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:30.995253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:30.996601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:30.997025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:30.997314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:30.998321 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.000135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.000804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:31.001373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.001373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:31.001871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.002609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.003334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:31.003334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.003334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:31.138224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:31.138906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:31.139157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:31.139305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:31.140279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:31.140663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.141711 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:31.143115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.144074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.144508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:31.145184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:31.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.146398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:31.146997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.147656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.147656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:31.147656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.147656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:31.231496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:31.236985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:31.237234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:31.237500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:31.238626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:31.239307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.240604 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:31.242072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.242589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.243540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.244413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:31.244413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.244413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.244800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:31.245388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.245947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.245947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:31.245947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.245947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:31.325115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:31.325533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:31.325680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:31.325979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:31.326726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:31.327181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:31.327485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.328436 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.329809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:31.330502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.331115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:31.331631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:31.332479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:31.333352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:31.333352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:35.803522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:35.804549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:35.804549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:35.805818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:35.824768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=248 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:46:35.842402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:35.842844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:35.844163 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:35.845421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:35.846152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:35.846806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:35.847802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:35.849207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:35.849207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:35.849207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:35.849207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:35.849207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:35.849207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:35.995588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:35.996440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:35.996647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:35.998080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:35.998781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:35.998781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.000108 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.001651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.002748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:36.003369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.003369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:36.003525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.004213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.004998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:36.004998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.004998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:36.138882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:36.139382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:36.139526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:36.139709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:36.140911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:36.141471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.142702 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:36.144112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.145144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.145578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:36.146227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.146865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.146865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:36.146865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.146865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.147080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:36.148144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.149390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.149390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:36.149390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.149390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:36.237262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:36.237799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:36.237959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:36.238170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:36.239112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:36.239546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.240577 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:36.254485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.254956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:36.255419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:36.255858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.256403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.257050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:36.325873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:36.326317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:36.326465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:36.326765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:36.327425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:36.327984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:36.328288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.329183 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.330585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:36.331034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.331423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:36.332278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:36.332794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:36.333336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:36.333336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:40.804833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:40.805621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:40.805762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:40.806661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:40.840175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:40.840572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:40.841526 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:40.842720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:40.843714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:40.844628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:40.845685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:40.846503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:40.846503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:40.846503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:40.846503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:40.846503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:40.846503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:40.996701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:40.997277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:40.997477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:40.998677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:40.998985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:40.999300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.000347 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.001914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.002341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:41.002725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.002725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:41.003688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.004176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.004757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:41.004757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.004757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:41.139636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:41.140279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:41.140399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:41.140480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:41.141357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:41.141666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.142186 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:41.143073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.143972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:41.144732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.145369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:41.145994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.147072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:41.237819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:41.238592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:41.238592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:41.238704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:41.239515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:41.239834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.240497 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:41.241685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.242638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:41.243115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.243501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.243732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:41.244241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.244732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.244732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:41.244732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.244732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:41.326646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:41.327169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:41.327345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:41.327592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:41.328146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:41.328552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:41.328793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.330153 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.332124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:41.332743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.333298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:41.333896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:41.334715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:41.335606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:41.335606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:45.494340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:45.495525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:45.495525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:45.504829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 02:46:45.505043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 02:46:45.505139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:46:45.541181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.541776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.542499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.542769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.544474 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.545568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.546207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.546613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.546613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.546613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.546613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Fixing resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:46:45.580638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, resource={'id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['97900556-2eeb-4240-89bd-ed8258b9022d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:48Z', 'updated_at': '2026-04-13T02:30:49Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9202}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.581179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.581776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8120b9cd-5d7c-4d4d-9a57-22a6c716f312, external_ids_update=None, columns={'tag': 9202}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.582083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.583109 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Failed to fix resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.584399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.585385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.585385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.585385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.585385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:46:45.616119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.616384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.616667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.616767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.617386 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.618673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.618966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.619279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.619279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.619279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.619279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:46:45.647550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.648028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.648733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.649169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.650035 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.651042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.651581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.652214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.652214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.652214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.652214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Fixing resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:46:45.730616 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, resource={'id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'enable_snat': True}, 'gw_port_id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-exgz1', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'ip_address': '172.24.5.236', 'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, {'ip_address': '2001:db8::2c9', 'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 3, 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46'}, resource_type=routers, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.730956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.731139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.731313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.731429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.732377 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.732747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Unable to update router 530b037d-a47e-4f63-bd55-041139eeee2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Failed to fix resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 13 02:46:45.734652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.735249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.735793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Fixing resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:46:45.805877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:45.812827 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:45.815207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:45.818678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:45.889843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.890331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.892751 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.078 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.895520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.895942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.896499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.896499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:45.897712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.898510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.899389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:45.899389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.899389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:45.900488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, resource={'id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'name': '', 'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'project_id': '', 'mac_address': 'fa:16:3e:92:a4:24', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000165409', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.900806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.236/24', '2001:db8::2c9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.901193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3ec69888-b9c9-4830-9d48-853dd81cf68b, lrouter_port=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.901590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.902710 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Maintenance task: Failed to fix resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.904170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:45.904643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:45.905068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:45.905068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:45.905068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:45.905068 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_for_inconsistencies finished in 0.409 seconds
Apr 13 02:46:45.997465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:45.998305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:45.998450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:45.999372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.999632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:45.999899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.000716 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.001683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.002435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:46.002833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:46.003282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.003784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:46.003784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.003784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.003784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:46.003784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.003784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:46.140479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:46.140876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:46.141036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:46.141213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:46.142239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:46.142630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.143605 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:46.145090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.145812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.146562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:46.147142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.147664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:46.148288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.149086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:46.238505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:46.239161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:46.239292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:46.239490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:46.240727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:46.241137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.242559 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:46.243915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.244708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.246156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.246761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:46.246761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.246761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.247371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:46.248204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.249058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.249058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:46.249058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.249058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:46.328036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:46.328733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:46.328886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:46.329198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:46.330238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:46.330812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:46.331101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.332023 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.333725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:46.334436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.335043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:46.335362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:46.336163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:46.336995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:46.336995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:50.813044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:50.814226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:50.814226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:50.815202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:50.855873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:50.856215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:50.857319 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:50.858527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:50.859238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:50.860046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:50.860874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:50.861832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:50.861832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:50.861832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:50.861832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:50.861832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:50.861832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:50.998574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:50.999637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:50.999891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:51.001171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:51.001540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:51.001790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.002699 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.004569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.005209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:51.005823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.005823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:51.006666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.007490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.008446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:51.008446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.008446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:51.141432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:51.142164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:51.142260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:51.142419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:51.142947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:51.143217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.143848 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:51.145200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.146329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:51.147569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.148538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:51.149529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.150394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:51.239339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:51.240190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:51.240389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:51.240636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:51.241791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:51.242172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.242980 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:51.244491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.245038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.245902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.246304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.246701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:51.246701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.246701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.247679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:51.248270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.248839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.248839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:51.248839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.248839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:51.328718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:51.329234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:51.329377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:51.329660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:51.330182 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:51.330457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:51.330631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.331362 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.332998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:51.333504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:51.334758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:51.335334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:51.335885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:51.335885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:51.335885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:55.813876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:55.815273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:55.815273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:55.816732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:55.850775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:55.851227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:55.852204 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:55.853136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:55.853834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:55.854552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:55.855441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:55.856672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:55.856672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:55.856672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:55.856672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:55.856672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:55.856672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:55.999583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:56.000164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:56.000304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:56.001343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:56.001590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:56.001813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.002591 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.003637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.004108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:56.004452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.004452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:56.004726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.005446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.006062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:56.006062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.006062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:56.142324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:56.142927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:56.143052 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:56.143161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:46:56.144130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:56.144578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.145359 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:56.146470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.147306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:56.147912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.148303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:56.148664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.149227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:56.240407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:56.241180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:56.241340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:56.241566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:56.242664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:56.243135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.244079 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:46:56.244977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.246077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:56.246949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:56.247896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.248442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.248880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:46:56.329504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:46:56.330079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:46:56.330207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:46:56.330446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:46:56.331349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:56.331916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:46:56.332682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.333547 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.334958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:46:56.335941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:46:56.336782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:46:56.337976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:46:56.339271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:46:56.339271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:46:56.339271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:00.814606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:00.815427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:00.815614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:00.816865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:00.850751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:00.851587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:00.852556 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:00.853664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:00.854621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:00.855547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:00.856456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:00.857417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:00.857417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:00.857417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:00.857417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:00.857417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:00.857417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:01.000514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:01.001186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:01.001385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:01.002727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:01.003134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:01.003790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.004414 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.006250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.007110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:01.007715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:01.008512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.009452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:01.009452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.009452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.009452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:01.009452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.009452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:01.143163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:01.143803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:01.143953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:01.144165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:01.145127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:01.145484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.146504 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:01.147879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.148756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.149333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:01.150056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.150717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.150717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:01.150717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.150717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.151194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:01.151694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.152400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.152400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:01.152400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.152400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:01.241349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:01.241922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:01.242123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:01.242300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:01.243094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:01.243517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.244377 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:01.245191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.245936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.246374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.246961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:01.247603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:01.248367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:01.330089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:01.330471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:01.330602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:01.330840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:01.331637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:01.331949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:01.332161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.332861 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.334515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:01.335237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.335876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:01.336103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:01.336966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:01.337881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:01.337881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:05.815382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:05.816533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:05.816533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:05.817554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:05.853369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:05.853846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:05.855108 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:05.856301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:05.857112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:05.857654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:05.858319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:05.859148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:05.859148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:05.859148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:05.859148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:05.859148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:05.859148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:06.001235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:06.001788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:06.001861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:06.002631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:06.002848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:06.002969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.003522 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.004250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.004786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:06.005277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:06.005786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.006438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:06.006438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.006438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.006438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:06.006438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.006438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:06.143974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:06.144511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:06.144966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:06.144966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:06.145573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:06.145909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.146696 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:06.148189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.148996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.149486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:06.150079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.150660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.150660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:06.150660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.150660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.151184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:06.151921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.152811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.152811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:06.152811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.152811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:06.242249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:06.242835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:06.242979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:06.243287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:06.244211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:06.244552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.245436 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:06.246918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.247845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.248256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.248801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:06.250057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:06.250560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.251093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:06.330796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:06.331551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:06.332208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:06.332208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:06.332589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:06.332716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:06.332820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.333640 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.334695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:06.335420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:06.336045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:06.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:06.337759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:06.337759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:06.337759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:10.816197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:10.816910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:10.817209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:10.818587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:10.854098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:10.854382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:10.855316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:10.856547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:10.857173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:10.857855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:10.858549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:10.859826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:10.859826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:10.859826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:10.859826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:10.859826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:10.859826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:11.002030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:11.002888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:11.003071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:11.004435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:11.004919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:11.005590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.006523 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.008298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.008775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:11.009164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.009164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:11.009980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.010517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.011174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:11.011174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.011174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:11.144783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:11.145410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:11.145577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:11.145739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:11.146675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:11.147052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.148025 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:11.149525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.150312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.150955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:11.151598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.152246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.152246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:11.152246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.152246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.152655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:11.153486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.154395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.154395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:11.154395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.154395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:11.243046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:11.243663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:11.243663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:11.243663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:11.244856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:11.245212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.246118 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:11.247099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.247637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.247881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.248398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:11.248810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:11.249284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.249812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:11.331512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:11.331887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:11.331993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:11.332300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:11.332932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:11.333364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:11.333616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.334489 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.336088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:11.336703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.337303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:11.337607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:11.338403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:11.339277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:11.339277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:15.817077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:15.818046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:15.818046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:15.819394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:15.853910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:15.854508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:15.855239 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:15.856081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:15.856633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:15.857677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:15.858879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:15.859757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:15.859757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:15.859757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:15.859757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:15.859757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:15.859757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:16.003176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:16.003755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:16.003755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:16.005155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:16.005563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:16.005842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.006875 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.008656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.009129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:16.009530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.009530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:16.010421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.010995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.011817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:16.011817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.011817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:16.145340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:16.146068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:16.146237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:16.146369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:16.147346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:16.147736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.148655 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:16.150108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.151359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.151877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:16.152742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.153272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.153272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:16.153272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.153272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.153430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:16.154132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.154867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.154867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:16.154867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.154867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:16.243965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:16.244361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:16.244465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:16.244651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:16.245339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:16.245771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.246664 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:16.247791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.248745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.249260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.250104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:16.250780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:16.251385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.252087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:16.332058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:16.332638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:16.332740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:16.332870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:16.333451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:16.333837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:16.334140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.334885 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:16.336693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:16.337454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:16.338510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:16.339657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:16.339657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:16.339657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:20.817715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:20.819275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:20.819474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:20.821166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:20.866258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:20.866448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:20.867655 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:20.868929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:20.869640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:20.870286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:20.871105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:20.872038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:20.872038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:20.872038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:20.872038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:20.872038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:20.872038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:21.004149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:21.005045 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:21.005045 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:21.006587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:21.006964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:21.007281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.008386 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.010221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.011056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:21.011606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.011606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:21.011766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.012792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.013396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:21.013396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.013396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:21.146218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:21.146900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:21.147042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:21.147110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:21.147986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:21.148348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.149380 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:21.150458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.151265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.151665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:21.152271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.152925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:21.153714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.154584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:21.244738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:21.245333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:21.245480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:21.245724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:21.246644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:21.247184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.248121 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:21.249521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.250412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.250854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.251403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.251950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:21.251950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.251950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.252537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:21.253320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.254491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.254491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:21.254491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.254491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:21.332909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:21.333737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:21.333976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:21.334376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:21.335126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:21.335519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:21.335796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.336789 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.338486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:21.339147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.339716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:21.340027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:21.340806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:21.341683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:21.341683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:25.819929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:25.820940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:25.820940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:25.821851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:25.864678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:25.865018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:25.866027 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:25.866935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:25.867630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:25.868310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:25.869156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:25.870174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:25.870174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:25.870174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:25.870174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:25.870174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:25.870174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:26.004890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:26.005689 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:26.005958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:26.007445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:26.007903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:26.008361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.009316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.011397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.012203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:26.012817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.012817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:26.013037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.013601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.014139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:26.014139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.014139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:26.147385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:26.147835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:26.147989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:26.148193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:26.149178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:26.149572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.150572 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:26.151985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.152549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.153369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:26.153774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.154182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.154182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:26.154182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.154182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.155065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:26.155717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.156388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.156388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:26.156388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.156388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:26.245546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:26.246202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:26.246341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:26.246517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:26.247239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:26.247697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.248565 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:26.249582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.250355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.250796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.251458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.252204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:26.252204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.252204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.252402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:26.253245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.254097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.254097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:26.254097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.254097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:26.333639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:26.334179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:26.334291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:26.334621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:26.335190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:26.335588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:26.335829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.336630 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.337822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:26.338300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.338638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:26.338835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:26.339387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:26.339863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:26.339863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:30.820599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:30.821668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:30.821799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:30.823371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:30.862129 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:30.862651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:30.863875 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:30.865084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:30.866086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:30.866877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:30.867991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:30.869276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:30.869276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:30.869276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:30.869276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:30.869276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:30.869276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:31.006348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:31.006935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:31.007169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:31.008565 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:31.009030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:31.009290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.010496 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.012120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.012786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:31.013370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.013370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:31.014055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.014580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.015415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:31.015415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.015415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:31.148036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:31.148780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:31.148992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:31.149246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:31.150024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:31.150516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.151543 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:31.152950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.153502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.154451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:31.155072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.155777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.155777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:31.155777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.155777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.156322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:31.157202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.158253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.158253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:31.158253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.158253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:31.246430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:31.247231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:31.247715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:31.247715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:31.248776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:31.249313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.250471 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:31.251583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.252495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:31.253094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.253614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.254149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:31.254801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:31.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:31.334771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:31.335479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:31.335590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:31.335880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:31.336625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:31.337079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:31.337324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.338211 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.339803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:31.340862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:31.341706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:31.342888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:31.344065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:31.344065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:31.344065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:35.821478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:35.822447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:35.822447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:35.823862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:35.832986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=427 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:47:35.871742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:35.872285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:35.873369 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:35.874539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:35.875131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:35.875680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:35.876362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:35.877183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:35.877183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:35.877183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:35.877183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:35.877183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:35.877183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:36.008158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:36.008158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:36.008530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:36.009874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:36.010130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:36.010827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.011408 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.012425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.013097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:36.013833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:36.014602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.015652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:36.015652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.015652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.015652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:36.015652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.015652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:36.148886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:36.149920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:36.150093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:36.150255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:36.151541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:36.151856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.152955 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:36.154541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.154966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.156069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:36.156408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.156772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.156772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:36.156772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.156772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.158042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:36.158478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.158966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.158966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:36.158966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.158966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:36.247207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:36.247677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:36.247793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:36.247939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:36.248485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:36.248820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.249790 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:36.251104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.251970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.252661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.253096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.253481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:36.253481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.253481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.254897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:36.255469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.255987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.255987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:36.255987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.255987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:36.335575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:36.336155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:36.336286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:36.336598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:36.337392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:36.337694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:36.337862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.338597 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.340160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:36.340872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.341534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:36.341757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:36.342605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:36.343538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:36.343538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:40.822367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:40.823223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:40.823304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:40.824913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:40.863193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:40.863640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:40.864774 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:40.866122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:40.866861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:40.867591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:40.868666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:40.869769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:40.869769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:40.869769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:40.869769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:40.869769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:40.869769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:41.008362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:41.008882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:41.008882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:41.010189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:41.010674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:41.011162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.012153 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.013323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.013854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:41.014237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.014237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:41.014533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.015093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.015657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:41.015657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.015657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:41.149996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:41.150468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:41.150584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:41.150764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:41.151682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:41.152078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.152958 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:41.154443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.155377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.155783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:41.156401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.157066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.157066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:41.157066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.157066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.157267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:41.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.159068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.159068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:41.159068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.159068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:41.247834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:41.248610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:41.248767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:41.248957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:41.249659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:41.250126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.250884 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:41.252478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.253230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.254028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.254585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.255213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:41.255213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.255213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.255830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:41.256614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.257446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.257446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:41.257446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.257446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:41.336203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:41.336836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:41.337026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:41.337309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:41.337857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:41.338243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:41.338485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.339425 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.341090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:41.341481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.341813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:41.342904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:41.343724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:41.344544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:41.344544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:45.823231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:45.823994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:45.824173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:45.825516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:45.861407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:45.861852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:45.863150 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:45.863845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:45.864522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:45.865214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:45.866082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:45.867061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:45.867061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:45.867061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:45.867061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:45.867061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:45.867061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:46.009136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:46.009842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:46.010096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:46.011077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:46.011370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:46.011606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.012770 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.014128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.014784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:46.015408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:46.016159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.017051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:46.017051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.017051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.017051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:46.017051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.017051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:46.150912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:46.151379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:46.151505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:46.151633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:46.152654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:46.153277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.154376 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:46.155652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.156512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.157126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:46.157754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.158342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.158342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:46.158342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.158342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.158622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:46.159385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.160233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.160233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:46.160233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.160233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:46.248783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:46.249854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:46.249854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:46.250167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:46.250781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:46.251184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.251726 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:46.253757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.254920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.256255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.256895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.257683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:46.257683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.257683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.257841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:46.258654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.259055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.259055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:46.259055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.259055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:46.336861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:46.337456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:46.337554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:46.337779 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:46.338401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:46.338845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:46.339129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.339942 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.340815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:46.341482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.342065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:46.342581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:46.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:46.344203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:46.344203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:50.824364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:50.825371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:50.825530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:50.826948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:50.866565 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:50.866929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:50.867824 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:50.868554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:50.869280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:50.869943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:50.870789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:50.871920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:50.871920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:50.871920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:50.871920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:50.871920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:50.871920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:51.010139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:51.010916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:51.011142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:51.012623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:51.013020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:51.013322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.014390 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.016218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.017094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:51.017636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.017636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:51.018040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.018740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.019231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:51.019231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.019231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:51.151544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:51.151893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:51.151969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:51.152141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:51.153029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:51.153607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.154547 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:51.155588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.156482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:51.157188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.157762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:51.158360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.159219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:51.249650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:51.250263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:51.250400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:51.250588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:51.251338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:51.251688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.252650 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:51.254159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.254930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.255521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.256217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.256814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:51.256814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.256814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.257325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:51.257820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.258357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.258357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:51.258357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.258357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:51.337621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:51.337967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:51.338188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:51.338438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:51.339172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:51.339576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:51.339811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.340692 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.342283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:51.342968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.343583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:51.343807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:51.344734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:51.345919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:51.345919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:55.825401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:55.826296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:55.826296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:55.826735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:55.866868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:55.867333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:55.868323 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:55.869342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:55.870157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:55.870963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:55.871882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:55.873037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:55.873037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:55.873037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:55.873037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:55.873037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:55.873037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:56.011363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:56.012310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:56.012638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:56.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:56.014529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:56.014855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.016079 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.017918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.018613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:56.019291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.019291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:56.019571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.020330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.021246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:56.021246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.021246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:56.152320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:56.152912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:56.153092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:56.153256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:47:56.154236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:56.154601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.155582 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:56.156577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.157459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:56.158215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.158849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:56.159492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.160366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:56.250515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:56.251108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:56.251253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:56.251434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:56.252091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:56.252470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.253428 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:47:56.254923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.255679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.256396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.256986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.257786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:56.257786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.257786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.258253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:56.259172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.259945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.259945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:56.259945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.259945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:47:56.338395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:47:56.338948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:47:56.339086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:47:56.339210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:47:56.339925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:56.340338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:47:56.340581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.341424 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.342367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:47:56.343167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:47:56.343799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:47:56.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:47:56.345531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:47:56.345531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:47:56.345531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:00.826073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:00.828356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:00.828356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:00.829373 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:00.873493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:00.874221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:00.875248 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:00.876528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:00.877681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:00.878384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:00.879219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:00.880221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:00.880221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:00.880221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:00.880221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:00.880221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:00.880221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:01.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:01.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:01.045549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:01.048277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:01.048277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:01.048509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.049929 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.051168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.051746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:01.052362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:01.053092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.053885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:01.053885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.053885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.053885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:01.053885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.053885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:01.153356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:01.153829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:01.153933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:01.154205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:01.155324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:01.155693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.156665 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:01.158474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.159490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.159936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:01.160639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.161394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.161394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:01.161394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.161394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.161640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:01.162160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.162679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.162679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:01.162679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.162679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:01.251598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:01.252471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:01.252706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:01.252898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:01.253674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:01.254034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.255034 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:01.256860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.257606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.258130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.258679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:01.259302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:01.260080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.260930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:01.339263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:01.339890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:01.340325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:01.340325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:01.341169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:01.341530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:01.341805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.342812 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.344373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:01.345046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.345668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:01.345959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:01.346806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:01.347639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:01.347639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:05.828580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:05.829932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:05.829932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:05.831081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:05.872140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:05.872714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:05.873891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:05.874830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:05.875507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:05.876216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:05.877068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:05.878107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:05.878107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:05.878107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:05.878107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:05.878107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:05.878107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:06.013072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:06.013759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:06.013882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:06.014698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:06.014893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:06.015162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.015831 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.016843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.017513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:06.018121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:06.018820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:06.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:06.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.019706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:06.153964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:06.154301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:06.154385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:06.154445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:06.155379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:06.155723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.156657 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:06.157534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.158413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:06.159115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.159858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:06.160510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.161422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:06.252401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:06.253057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:06.253257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:06.253435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:06.254466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:06.254885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.255837 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:06.257294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.258065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.258643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.259256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.259855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:06.259855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.259855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.260462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:06.261023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.261604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.261604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:06.261604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.261604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:06.339969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:06.340374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:06.340538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:06.340793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:06.341787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:06.342220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:06.342453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.343382 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.344943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:06.345703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.346378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:06.346846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:06.347815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:06.348942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:06.348942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:10.832423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:10.835255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:10.835255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:10.842178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:10.886570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:10.887049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:10.888273 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:10.889755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:10.890086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:10.890320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:10.890320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:10.891312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:10.891976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:10.892958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:10.892958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:10.892958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:11.014589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:11.015694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:11.015898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:11.017197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:11.017558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:11.017884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.018892 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.020286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.020987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:11.021860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.021860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:11.022775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.023299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.023784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:11.023784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.023784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:11.156685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:11.160272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:11.160431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:11.160592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:11.161447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:11.161823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.162747 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:11.164133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.164617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.165835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:11.166234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.166632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.166632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:11.166632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.166632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.168750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:11.169341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.169959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.169959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:11.169959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.169959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:11.252904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:11.253166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:11.253241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:11.253365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:11.254271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:11.254618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.255644 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:11.257640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.258136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.260328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.260683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.261105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:11.261105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.261105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.261553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:11.262058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.262566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.262566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:11.262566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.262566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:11.341051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:11.341310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:11.341462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:11.341732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:11.342559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:11.343041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:11.343313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.344231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.345975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:11.346892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:11.347585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:11.348400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:11.349416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:11.349416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:11.349416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:15.835996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:15.837109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:15.837329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:15.839669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:15.883036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:15.883551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:15.884625 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:15.885845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:15.886567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:15.887247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:15.888328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:15.889391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:15.889391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:15.889391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:15.889391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:15.889391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:15.889391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:16.015823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:16.016529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:16.016676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:16.017761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:16.018028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:16.018235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.019016 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.020183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.020808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:16.021435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:16.022221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.023105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:16.023105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.023105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.023105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:16.023105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.023105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:16.160494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:16.161272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:16.161377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:16.161721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:16.162040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:16.162433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Traceback (most recent call last):
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.163115 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:16.163975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:16.165232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.165960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.166623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:16.167304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.168571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.169588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:16.253898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:16.254496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:16.254584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:16.254714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:16.255408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:16.255811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.256573 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:16.257639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.258475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:16.259166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:16.259795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.260454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.261432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:16.341386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:16.341975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:16.341975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:16.342198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:16.342681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:16.343115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:16.343362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.344198 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.345601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:16.346035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.346416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:16.346593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:16.347207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:16.347744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:16.347744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:20.836852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:20.837765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:20.837813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:20.839179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:20.878349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:20.878889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:20.880194 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:20.881761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:20.882218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:20.882556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:20.882556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:20.883563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:20.884091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:20.884595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:20.884595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:20.884595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:21.017765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:21.018647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:21.019068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:21.019936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:21.020294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:21.020559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.021599 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.023021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.023528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:21.023944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.023944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:21.024918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.025492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.026106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:21.026106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.026106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:21.161694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:21.162267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:21.162450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:21.162727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:21.163332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:21.163615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Traceback (most recent call last):
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.164453 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:21.166297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.167323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.168419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:21.168872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.169308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.169308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:21.169308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.169308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-63ad438b-0f66-4ca1-90ba-f50adeffc4db None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:21.170115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.170741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.171327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:21.285631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:21.285631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:21.285631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:21.285631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:21.286743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:21.287557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.288279 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:21.289648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.290682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:21.291684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:21.292314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.292726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.293286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:21.342340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:21.343164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:21.343257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:21.343491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:21.344369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:21.344831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:21.345140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.346243 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.347884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:21.348419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.348825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:21.349665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:21.350182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:21.350658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:21.350658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:25.838298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:25.839301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:25.839301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:25.839965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:25.871616 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:25.872025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:25.872950 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:25.873793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:25.874193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:25.874565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:25.875053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:25.875590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:25.875590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:25.875590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:25.875590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:25.875590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:25.875590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:26.018622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:26.019238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:26.019448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:26.020833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:26.020922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:26.021190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.021787 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.023496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.023888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:26.024287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.024287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:26.025299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.025956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.026639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:26.026639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.026639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:26.162300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:26.163356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:26.163470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:26.163617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:26.164499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:26.164859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.165818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:26.167025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.167407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.168446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:26.168725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.169154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.169154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:26.169154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.169154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:26.170209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.170550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.170985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:26.285251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:26.285836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:26.285986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:26.286177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:26.286716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:26.287156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.287908 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:26.289054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.289796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.290377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.290963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.291765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:26.291765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.291765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.291927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:26.292737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.293464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.293464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:26.293464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.293464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:26.343147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:26.343529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:26.343685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:26.343854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:26.344247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:26.344581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:26.344785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.345486 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.347170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:26.347491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.347776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:26.348100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:26.348512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:26.349053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:26.349053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:30.839048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:30.839805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:30.839967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:30.841075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:30.872343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:30.872651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:30.873455 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:30.874376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:30.875083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:30.875753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:30.876589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:30.877627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:30.877627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:30.877627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:30.877627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:30.877627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:30.877627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:31.019308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:31.020103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:31.020321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:31.021341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:31.021590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:31.021788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.022484 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.024215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.024813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:31.025467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:31.026167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.027005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:31.027005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.027005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.027005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:31.027005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.027005 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:31.163083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:31.163691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:31.163889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:31.164135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:31.165054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:31.165485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Traceback (most recent call last):
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.166505 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:31.167883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.168712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.169262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:31.169892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.170532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.170532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:31.170532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.170532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-dd548fb2-a3cd-4354-894e-b20109b4a379 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:31.171208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.172159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.172988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:31.286210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:31.286982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:31.287139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:31.287320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:31.288304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:31.288725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.289674 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:31.291176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.291876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.292602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.292981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.293415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:31.293415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.293415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:31.295127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.295790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.295790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:31.295790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.295790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:31.343955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:31.344660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:31.344797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:31.345116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:31.345682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:31.346159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:31.346700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.347275 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.348521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:31.349242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:31.349830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:31.352128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:31.353571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:31.353571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:31.353571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:35.839847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:35.840783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:35.840783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:35.842116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:35.844213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=239 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:48:35.882483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:35.882838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Traceback (most recent call last):
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:35.883694 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:35.884667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:35.885145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:35.885561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:35.885561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:35.885746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:35.886414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:35.886938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:35.886938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:35.886938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:36.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:36.020672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:36.020806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:36.022308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:36.022612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:36.022880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.023899 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.025654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.026278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:36.026856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.026856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:36.027468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.028118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.028824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:36.028824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.028824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:36.164364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:36.165392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:36.165691 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:36.165993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:36.167374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:36.167718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.168811 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:36.170438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.171052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.171778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:36.172236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.172686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.172686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:36.172686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.172686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:36.173595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.174090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.174578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:36.287236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:36.287551 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:36.287697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:36.287864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:36.289016 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:36.289381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.290098 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:36.292172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.292656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.293719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.294308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:36.294909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:36.295685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.296498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:36.344988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:36.348996 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.349699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.350629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:36.351483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:36.352500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:36.353747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:36.353747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:36.353747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:36.353747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:36.353747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:36.353747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:40.840601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:40.841228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:40.841353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:40.842673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:40.877955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:40.878480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:40.879681 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:40.880985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:40.881670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:40.882386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:40.882386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:40.882540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:40.883400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:40.884404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:40.884404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:40.884404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:41.021125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:41.022091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:41.022091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:41.023345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:41.023718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:41.023967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.024925 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.026104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.026839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:41.027453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:41.028326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.029218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:41.029218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.029218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.029218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:41.029218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.029218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:41.165323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:41.165865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:41.166092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:41.166293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:41.167319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:41.167721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.168756 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:41.170380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.171344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.171987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:41.172690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.173266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.173266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:41.173266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.173266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:41.173540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.174075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.174797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:41.287940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:41.288633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:41.288735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:41.288851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:41.289756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:41.290236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.291124 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:41.292125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.292661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.292973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.293369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.293796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:41.293796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.293796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.294029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:41.294530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.295100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.295100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:41.295100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.295100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:41.346245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:41.346886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:41.346965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:41.347227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:41.347856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:41.348264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:41.348538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.349391 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.350733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:41.351176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.351659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:41.352319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:41.354075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:41.355233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:41.355233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:45.841431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:45.842253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:45.842253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:45.843668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:45.879484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:45.879898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Traceback (most recent call last):
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:45.880958 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:45.881802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:45.882292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:45.882711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:45.883249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:45.883881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:45.883881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:45.883881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:45.883881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:45.883881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:45.883881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:46.021910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:46.022676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:46.022808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:46.023843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:46.024309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:46.024592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.025349 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.026975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.027592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:46.028252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.028252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:46.028797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.029523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.030364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:46.030364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.030364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:46.166122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:46.166814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:46.167115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:46.167292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:46.168072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:46.168658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.169701 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:46.171403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.172467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.173154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:46.173752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:46.174246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.174642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.175176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:46.288750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:46.289718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:46.289916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:46.290216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:46.291040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:46.291393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.292370 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:46.293844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.294695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.295188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.295932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:46.296828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:46.297581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.298108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:46.347160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:46.347830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:46.347972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:46.348276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:46.349196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:46.349614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:46.349815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.350514 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.352036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:46.352658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.353289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:46.353399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:46.354162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:46.354890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:46.354890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:50.842238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:50.843150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:50.843373 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:50.844856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:50.881773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:50.882307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:50.883358 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:50.884670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:50.885346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:50.886050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:50.886787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:50.887776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:50.887776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:50.887776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:50.887776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:50.887776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:50.887776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:51.022887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:51.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:51.023537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:51.024704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:51.025035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:51.025371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.026663 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.028185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.028799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:51.029380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.029380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:51.029951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.030697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.031599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:51.031599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.031599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:51.166817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:51.167547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:51.167695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:51.167768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:51.168517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:51.168918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.169915 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:51.170992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.171792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.172295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:51.172890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:51.173369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.173753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.174286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:51.289671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:51.290236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:51.290379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:51.290570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:51.291486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:51.291822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.292798 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:51.294321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.295080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.295768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.296421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.297268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:51.297268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.297268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.297662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:51.298197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.298757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.298757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:51.298757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.298757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:51.348150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:51.348829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:51.348974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:51.349280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:51.350092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:51.350463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:51.350712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.351691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.353364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:51.354171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:51.354776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:51.355611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:51.356537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:51.356537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:51.356537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:55.843217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:55.844352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:55.844352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:55.845219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:55.884729 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:55.885187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Traceback (most recent call last):
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:55.886197 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:55.887167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:55.887875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:55.888694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:55.889574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:55.890757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:55.890757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:55.890757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:55.890757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:55.890757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:55.890757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:56.023371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:56.024024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:56.024147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:56.025189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:56.025535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:56.025963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.026639 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.027915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.028637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:56.029289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:56.030066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.030995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:56.030995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.030995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.030995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:56.030995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.030995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:56.167603 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:56.168357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:56.168620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:56.168824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:48:56.169522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:56.169833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.170734 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:56.172234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.174017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:56.174305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.174755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.174755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:56.174755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.174755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:56.176565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.176908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.177286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:56.290471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:56.291336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:56.291528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:56.291788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:56.292563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:56.293062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.294403 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:48:56.295448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.296281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.296762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:56.297227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:56.297579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:56.297979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.298524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:48:56.348879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:48:56.349719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:48:56.349888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:48:56.350204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:48:56.350844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:56.351328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:48:56.351568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.352486 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.354206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:48:56.354814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.355362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:48:56.355857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:48:56.356776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:48:56.357361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:48:56.357361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:00.843950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:00.844600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:00.844804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:00.846095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:00.881574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:00.882026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:00.883029 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:00.884186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:00.884818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:00.885496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:00.886300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:00.887309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:00.887309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:00.887309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:00.887309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:00.887309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:00.887309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:01.024319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:01.025033 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:01.025287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:01.026390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:01.026754 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:01.027029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.027931 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.029584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.030288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:01.030916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.030916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:01.031319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.032073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.032867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:01.032867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.032867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:01.168309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:01.168856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:01.168969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:01.169236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:01.170117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:01.170478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.171461 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:01.173104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.173849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.174806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:01.175443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.176058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.176058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:01.176058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.176058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:01.177109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.177687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.178201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:01.291563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:01.292236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:01.292344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:01.292516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:01.293374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:01.293825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.294754 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:01.295758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.296448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.296846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:01.297378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:01.297970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:01.298609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.299471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:01.349700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:01.350166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:01.350317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:01.350575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:01.351199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:01.351557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:01.351791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.352587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.354530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:01.355173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.355919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:01.356144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:01.356892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:01.357407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:01.357407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:05.844890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:05.846208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:05.846208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:05.847105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:05.884747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:05.885144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:05.885985 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:05.887150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:05.887770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:05.888635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:05.889426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:05.890408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:05.890408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:05.890408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:05.890408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:05.890408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:05.890408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:06.025211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:06.025949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:06.026170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:06.027310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:06.027704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:06.027949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.028928 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.030756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.031350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:06.031899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.031899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:06.032409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.033215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.034200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:06.034200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.034200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:06.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:06.170201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:06.170477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:06.170748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:06.171502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:06.171725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.172535 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:06.174154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.174946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.175875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:06.176505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.177120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.177120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:06.177120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.177120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:06.177826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.178682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.179599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:06.292590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:06.293330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:06.293516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:06.293783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:06.294799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:06.295295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Traceback (most recent call last):
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.296331 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:06.297733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.298464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.299204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:06.299762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.300456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.300456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:06.300456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.300456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ef443445-5fc2-440c-9c35-30a0727cfdf3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.300918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:06.301997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.302830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.302830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:06.302830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.302830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:06.350513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:06.350766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:06.350840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:06.351035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:06.351651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:06.352025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:06.352315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.353070 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.354260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:06.354924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.355596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:06.355806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:06.356603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:06.357439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:06.357439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:10.846808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:10.847240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:10.847240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:10.848627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:10.884713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:10.885202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:10.886230 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:10.887064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:10.887781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:10.889082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:10.889927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:10.891378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:10.891378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:10.891378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:10.891378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:10.891378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:10.891378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:11.025887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:11.026808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:11.027030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:11.028373 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:11.028722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:11.028958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.030060 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.031611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.032592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:11.033192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.033192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:11.033622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.034400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.035279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:11.035279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.035279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:11.170360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:11.170982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:11.171237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:11.171462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:11.172313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:11.172565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.173325 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:11.175052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.176187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.176912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:11.177650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:11.178518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.179552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.180684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:11.293617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:11.293959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:11.294074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:11.294195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:11.295181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:11.295672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.296729 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:11.297725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.298573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:11.299324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:11.300044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.300728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.301665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:11.351172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:11.351571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:11.351719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:11.352044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:11.352857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:11.353247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:11.353458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.354243 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.355436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:11.356271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:11.356858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:11.357655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:11.358615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:11.358615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:11.358615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:15.847290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:15.848095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:15.848095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:15.849456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:15.884118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:15.884729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:15.885797 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:15.886947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:15.887545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:15.888172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:15.888862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:15.889747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:15.889747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:15.889747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:15.889747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:15.889747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:15.889747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:16.026845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:16.027677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:16.027871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:16.029541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:16.030157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:16.030353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.032198 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.033931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.034339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:16.034690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.034690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:16.035664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.036358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.036882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:16.036882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.036882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:16.171167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:16.171809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:16.171925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:16.172094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:16.172895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:16.173326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.174311 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:16.175327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.176065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.176558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:16.177219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:16.177870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.178582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.179595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:16.294432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:16.295312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:16.295312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:16.295668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:16.296493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:16.297161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.297893 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:16.298856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.299736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.300570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.300927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.301359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:16.301359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.301359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.302559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:16.303212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.303727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.303727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:16.303727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.303727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:16.351809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:16.352387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:16.352524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:16.352769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:16.353394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:16.353808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:16.354246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.354856 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.356683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:16.357286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.357814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:16.358299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:16.359096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:16.359863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:16.359863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:20.847852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:20.848582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:20.848833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:20.850253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:20.889790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:20.890340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:20.891482 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:20.892578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:20.893306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:20.893985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:20.894809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:20.895784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:20.895784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:20.895784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:20.895784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:20.895784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:20.895784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:21.027693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:21.028287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:21.028444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:21.029925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:21.030401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:21.030671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.031829 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.033451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.034107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:21.034710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.034710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:21.035107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.035852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.036914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:21.036914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.036914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:21.172048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:21.172631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:21.172852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:21.173081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:21.174117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:21.174510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.175536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:21.177022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.178064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.178521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:21.179164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.179816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.179816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:21.179816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.179816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:21.180216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.180940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.181864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:21.295585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:21.296472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:21.296754 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:21.296918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:21.297924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:21.298411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.299419 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:21.300860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.301705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.302470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.303328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.303990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:21.303990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.303990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:21.305162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.306161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.306161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:21.306161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.306161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:21.352587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:21.353328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:21.353496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:21.353885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:21.354438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:21.354764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:21.354961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.355592 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.356497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:21.356895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.357269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:21.357529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:21.358096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:21.358640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:21.358640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:25.848664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:25.849477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:25.849675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:25.850991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:25.890048 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:25.890674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:25.891568 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:25.893253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:25.893792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:25.894357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:25.894357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:25.895154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:25.895898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:25.896893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:25.896893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:25.896893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:26.028588 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:26.029566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:26.029851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:26.031476 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:26.031921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:26.032233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.033500 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.035711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.036626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:26.037540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.037540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:26.038299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.039182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.039934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:26.039934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.039934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:26.172738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:26.173526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:26.173735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:26.173947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:26.174724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:26.175135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.176108 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:26.177384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.178269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.178687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:26.179316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.179926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.179926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:26.179926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.179926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:26.180513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.181280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.182124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:26.296436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:26.297140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:26.297245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:26.297438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:26.298534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:26.298891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.299753 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:26.301249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.301957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.302594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.303340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.304081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:26.304081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.304081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.304269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:26.305036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.305854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.305854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:26.305854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.305854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:26.353450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:26.354015 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:26.354207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:26.354542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:26.355357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:26.355826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:26.356108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.357104 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.358545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:26.358932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.359283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:26.360157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:26.360641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:26.361531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:26.361531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:30.849449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:30.897057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:30.897057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:30.898019 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:30.934728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:30.935107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:30.935921 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:30.936797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:30.937484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:30.938202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:30.939043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:30.940102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:30.940102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:30.940102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:30.940102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:30.940102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:30.940102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:31.327605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:31.327605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:31.327605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:31.327605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:31.327605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:31.328226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:31.329153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:31.329282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:31.330892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:31.330892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:31.330892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:31.334051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:31.334333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:31.334473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.335732 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.336688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.336978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:31.337240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.337240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:31.337240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:31.338268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.338617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.338971 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:31.339339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:31.339745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.340303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:31.340589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.340835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.340835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:31.340835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.340835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:31.340835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:31.341548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.341989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.342543 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:31.342988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.343598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.343844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.344376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:31.344799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:31.345424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.345991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:31.354055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:31.354705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:31.354962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:31.355057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:31.355627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:31.355870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:31.356077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.356770 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.358418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:31.359115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.359674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:31.360122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:31.361056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:31.361942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:31.361942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:35.896933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:35.897941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:35.897941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:35.899286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:35.901875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:49:35.938232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:35.938607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Traceback (most recent call last):
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:35.939641 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:35.940787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:35.941320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:35.941722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:35.941722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d5626746-6fda-4a20-83ee-6d40e92126f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:35.941869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:35.942386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:35.942948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:35.942948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:35.942948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:36.327706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:36.328625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:36.328625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:36.329778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:36.330396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:36.330996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:36.331416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:36.332474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:36.332926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:36.333797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.334884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:36.335646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.336298 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:36.336664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:36.336949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.338827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.339223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:36.339586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.339586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:36.339586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:36.339586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.341868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.342418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:36.342921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:36.343959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:36.344481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:36.346368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.346867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.347798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:36.348308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.348830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.348830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:36.348830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.348830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:36.349515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.349955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task update_ha_failover failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.351132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:36.351567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.351970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.351970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:36.351970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.351970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.353071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:36.353722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.354270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.354270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:36.354270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.354270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:36.354696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:36.355117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:36.355229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:36.355466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:36.355882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:36.356077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:36.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.356647 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.358057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:36.358542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.358936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:36.359621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:36.360256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:36.360839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:36.360839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:40.897812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:40.898646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:40.898736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:40.900197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:40.948276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:40.948564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Traceback (most recent call last):
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:40.949431 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:40.950860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:40.951462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-6f45f70b-f89a-4d8c-a7bb-2f9b2937058a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:40.952054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:40.952702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:40.953514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:40.953514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:40.953514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:40.953514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:40.953514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:40.953514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:41.328807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:41.330383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:41.330556 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:41.331922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:41.332773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:41.332773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:41.333217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:41.333569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:41.334296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:41.335316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:41.335836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:41.336209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:41.336746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:41.337115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.338159 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.339927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.340327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:41.340716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.340716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:41.340716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:41.341836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.342326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.342978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:41.342978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.342978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:41.342978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.343288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:41.344247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.346939 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.347640 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:41.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.349456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:41.351269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.351731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.016 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.352471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:41.353665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.354379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.354379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:41.354379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.354379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_ha_failover failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.354606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:41.355337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.356130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.356130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:41.356130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.356130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.356347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:41.357245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:41.358307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:41.359175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:41.360148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.360873 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.361716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:41.362068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.362415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:41.362966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:41.363383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:41.363816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:41.363816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:45.898585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:45.899680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:45.899680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:45.901161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:45.939653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:45.940053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:45.941220 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:45.942103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:45.942765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:45.943475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:45.944356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:45.945402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:45.945402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:45.945402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:45.945402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:45.945402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:45.945402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:46.329657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:46.330349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:46.330557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:46.331683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:46.331945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:46.332069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.332704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.332984 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:46.333466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.334834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.335486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:46.336042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.336042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:46.336042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:46.336042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.336974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.337842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:46.338464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.338464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:46.338464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:46.338464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:46.338464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:46.338464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:46.338872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:46.339126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.339889 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:46.340820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:46.341324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.342790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:46.343200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.343593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.343593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:46.343593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.343593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:46.343593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:46.345322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.345757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Traceback (most recent call last):
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.346322 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:46.346744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.347187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.347673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:46.348103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.348498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.348498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:46.348498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.348498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.349336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:46.349812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.350349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.350349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:46.350349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.350349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:46.358294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:46.359014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:46.359303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:46.359646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:46.360391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:46.360631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:46.360788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.361490 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.363357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.363744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:46.364224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.364224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:46.365262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:46.365724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:46.366253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:46.366253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:46.366253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:50.899353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:50.900378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:50.900378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:50.901644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:50.936646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:50.937180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:50.938191 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:50.939350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:50.940107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:50.940877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:50.941857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:50.942993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:50.942993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:50.942993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:50.942993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:50.942993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:50.942993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:51.330507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:51.331185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:51.331332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:51.332872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:51.333371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:51.333637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.334314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:51.334623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:51.334861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:51.335100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.335933 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.337448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.337853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:51.338249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.338249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.338916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.339445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:51.339949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.339949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:51.339949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:51.339949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:51.340125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:51.340674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.341160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:51.341280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:51.341964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:51.341964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.342116 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:51.343503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.344104 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:51.344711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.345351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.345608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:51.346034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.346425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.346425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:51.346425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.346425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:51.346549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.347062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.347610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.347817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.348207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.348632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:51.348632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.348632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.348754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:51.349278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.349789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.349789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:51.349789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.349789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:51.358737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:51.359317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:51.359677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:51.359985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:51.360717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:51.361412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:51.361789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.362721 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.363613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:51.364357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:51.364974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:51.365782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:51.366611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:51.366611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:51.366611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:55.900456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:55.901491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:55.901491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:55.902240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:55.950534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:55.950954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:55.951893 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:55.952705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:55.953363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:55.953984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:55.954787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:55.955746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:55.955746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:55.955746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:55.955746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:55.955746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:55.955746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:56.331304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:56.332112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:56.332231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:56.333282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:56.333621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:56.334244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.334751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:56.335246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.335622 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:56.335939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.337188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.337635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:56.338032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.338032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:56.338308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.338894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.339710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:56.340175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:56.340423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:56.340663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:56.341511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:56.341657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.342383 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Traceback (most recent call last):
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.342774 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:56.344390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.344930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:49:56.345437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.345888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.346548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:56.346932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.347367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.347367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:56.347367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.347367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.347605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:56.348056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.348460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.348460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:56.348460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.348460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.349016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:56.349516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.350072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.350072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:56.350072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.350072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ea3df1d2-2bde-47ed-a5c2-f7ac1e156a82 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:56.350374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.350836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.351353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:49:56.359315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:49:56.359934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:49:56.360117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:49:56.360382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:49:56.361132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:56.361582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:49:56.361669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.362487 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.364039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:49:56.364462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.364789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:49:56.365578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:49:56.366065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:49:56.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:49:56.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:00.901126 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:00.902232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:00.902232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:00.903067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:00.947557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:00.947949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Traceback (most recent call last):
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:00.948587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:00.949319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:00.950428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2a2c8b06-f21b-49cb-9524-9c856b869fc0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:00.951347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:00.953275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:00.954992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:00.954992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:00.954992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:00.954992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:00.954992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:00.954992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:01.332270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:01.333536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:01.333749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:01.335257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:01.335893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:01.336542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:01.337430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.337666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:01.337791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:01.338051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Traceback (most recent call last):
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.338966 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.341914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.342592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:01.343153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.343153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:01.343153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:01.343153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:01.344191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.344650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.345224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:01.345224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.345224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:01.345224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.345224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:01.345427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:01.345599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.345978 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:01.347488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.348030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:01.348517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task set_fip_distributed_flag failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.350407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:01.350799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.351199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.351199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:01.351199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.351199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:01.351199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:01.353524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.354023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Traceback (most recent call last):
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.354509 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:01.355520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.355964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] OVN maintenance task update_ha_failover failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.356944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:01.357795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.358229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.358229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:01.358229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.358229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-40a3277b-804b-4c7a-a0bf-b35a9632f38e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.358828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:01.359365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.359889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.359889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:01.359889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.359889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:01.359889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:01.360423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:01.360936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:01.360936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:01.361252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:01.361622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:01.361865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.362654 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.363711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:01.364141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.364493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:01.364710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:01.365238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:01.365714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:01.365714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:05.901846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:05.902634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:05.902634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:05.903853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:05.940741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:05.941270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:05.942309 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:05.943182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:05.943852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:05.944655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:05.945529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:05.947667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:05.947667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:05.947667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:05.947667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:05.947667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:05.947667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:06.333274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:06.333959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:06.334107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:06.335358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:06.335800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:06.336094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.337245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.338452 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.339134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.339675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:06.340652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.340652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:06.340652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:06.341410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.341845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.342386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:06.342386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.342386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:06.342386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:06.342386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:06.342538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:06.342642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.343561 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:06.345151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:06.345651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:06.347557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.347917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.347917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:06.347917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.347917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:06.347917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.349368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:06.349862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.350455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.350455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:06.350455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.350455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:06.350455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:06.350455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:06.350663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:06.350949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Traceback (most recent call last):
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.351865 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:06.353268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.353685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.354169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:06.354499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.354903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.354903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:06.354903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.354903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.355262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:06.355731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.356275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.356275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:06.356275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.356275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:06.360252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:06.360939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:06.361105 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:06.361354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:06.362131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:06.362508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:06.362780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Traceback (most recent call last):
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.363740 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.365126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.365719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c8c56985-a442-4f4e-b53f-27b702cc2af4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:06.366364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:06.367090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:06.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:06.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:06.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:06.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:06.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:06.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:10.902675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.010041 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.010859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.011723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:11.012794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:11.013673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.015114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:11.015114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.015114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.015114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:11.015114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.015114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:11.334204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:11.334953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:11.335146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:11.336378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.337288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.337591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.338614 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.340079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.340748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:11.341491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.341491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:11.341491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.341971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.342837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:11.343482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.343482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:11.343482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:11.343482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:11.343482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:11.343686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.343733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.344474 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:11.345956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:11.346484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.348255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:11.348647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.349071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.349071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:11.349071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.349071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:11.349071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:11.350491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.351112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:11.351741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.352102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.353037 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:11.353937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.354440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.354916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:11.355340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.355771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.355771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:11.355771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.355771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.356393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:11.357027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.357489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.357489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:11.357489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.357489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:11.360809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:11.361298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:11.361442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:11.361803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:11.362222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.362572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:11.362804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.363542 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.364818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:11.365287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.365680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:11.366596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:11.367413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:11.367977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:11.367977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:15.903568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:15.904637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:15.904637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:15.906474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:15.936155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:15.936689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:15.937842 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:15.938771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:15.939435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:15.939991 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:15.940701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:15.941597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:15.941597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:15.941597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:15.941597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:15.941597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:15.941597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:16.335225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:16.335909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:16.336154 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:16.337631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:16.338107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:16.338414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.339512 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.341332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.342043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:16.342728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.342728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:16.343421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.344383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:16.345315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.345820 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:16.347183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.347693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.348721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:16.349138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.349607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.349607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:16.349607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.349607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:16.349607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.351091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:16.351641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:16.352265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:16.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:16.352769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.353716 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:16.354713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.355197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.355669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:16.356035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.356479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.356479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:16.356479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.356479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.356807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:16.357424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.358022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.358022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:16.358022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.358022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:16.361319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:16.362021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:16.362229 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:16.362495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:16.362913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:16.363172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:16.363322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.363904 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.365480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:16.365933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.366337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:16.366808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:16.367386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:16.367913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:16.367913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:20.904738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:20.905307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:20.905393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:20.906698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:20.945453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:20.945453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:20.946539 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:20.947711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:20.948348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:20.948732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:20.948732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:20.948808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:20.949397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:20.950234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:20.950234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:20.950234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:21.336035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:21.336742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:21.336847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:21.338132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:21.338519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:21.338766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.339757 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.341541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.342255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:21.342878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.342878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:21.343230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.344060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.345189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:21.345189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.345189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:21.345189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:21.345189 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:21.345460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:21.345521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:21.346032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:21.346258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.346837 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:21.348394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.349265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.349863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:21.350263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.350616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.350616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:21.350616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.350616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.351856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:21.353318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:21.353759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:21.353986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:21.353986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.354843 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:21.356315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.356694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.357972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:21.358437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.358749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.358749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:21.358749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.358749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.359805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:21.360213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.360683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.360683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:21.360683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.360683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:21.361712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:21.362300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:21.362399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:21.362594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:21.363031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:21.363366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:21.363565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.364169 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.365312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:21.365722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.366103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:21.366694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:21.367224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:21.367742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:21.367742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:25.905368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:25.906662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:25.906662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:25.907713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:25.944781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:25.945307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:25.946461 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:25.947389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:25.947919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:25.948307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:25.948789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:25.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:25.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:25.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:25.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:25.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:25.949339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:26.337726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:26.337726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:26.338720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:26.339499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:26.339921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:26.340225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.341292 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.342514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.343073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:26.343465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.343465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:26.343720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.344269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.344842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:26.344842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.344842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:26.345156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:26.345701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:26.345793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:26.345898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:26.346406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:26.346652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.347280 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:26.348236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.348766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.349058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:26.349420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.349804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.349804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:26.349804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.349804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.350050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:26.350539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.351172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.351172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:26.351172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.351172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:26.352349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:26.352819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:26.353070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:26.353309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:26.353844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:26.354207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.355196 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:26.356671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.357143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.358222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:26.358570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.358993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.358993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:26.358993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.358993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.360015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:26.360518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.361065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.361065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:26.361065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.361065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:26.362123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:26.362872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:26.363098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:26.363439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:26.363956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:26.364258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:26.364452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.365135 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.366763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:26.367497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.368149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:26.368817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:26.369891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:26.370850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:26.370850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:30.906172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:30.906840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:30.907060 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:30.908384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:30.944321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:30.944779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:30.946028 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:30.946884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:30.947319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:30.947641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:30.947641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:30.947704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:30.948149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:30.948698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:30.948698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:30.948698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:31.337850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:31.338389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:31.338571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:31.339882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:31.340317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:31.340776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.341540 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.343274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.343977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:31.344621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.344621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:31.344948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.345669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:31.346474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:31.346835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:31.347104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.347620 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:31.348376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.349542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:31.349918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.350302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:31.350624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.351126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:31.352646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:31.353427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:31.353427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:31.353427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:31.354130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:31.354237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.354720 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:31.355658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.356468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.356897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:31.357513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:31.358448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:31.359189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.360862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:31.362736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:31.363628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:31.363788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:31.364086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:31.364681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:31.365066 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:31.365300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.366038 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:31.367494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.367853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:31.368128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:31.368593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:31.369119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:31.369119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:35.906930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:35.907750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:35.907750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:35.908855 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:35.911503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:50:35.943420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:35.944094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:35.944816 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:35.946106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:35.946604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:35.947028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:35.947028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:35.947200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:35.947786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:35.948514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:35.948514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:35.948514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:36.338763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:36.339288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:36.339407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:36.340817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:36.341299 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:36.341593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.342649 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.344518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.345251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:36.345893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:36.346666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:36.347735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:36.348318 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:36.348318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.348436 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:36.349841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.350776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.351294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:36.352029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.352749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.352749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:36.352749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.352749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.352988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:36.353472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.353980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.353980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:36.353980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.353980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:36.353980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:36.353980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:36.354213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:36.354443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:36.354781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:36.354977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.355538 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:36.356806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.357156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.358303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:36.358540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.358853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.358853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:36.358853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.358853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.360065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:36.360505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.360943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.360943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:36.360943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.360943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:36.363349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:36.363990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:36.364484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:36.364484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:36.364872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:36.365190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:36.365391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.365937 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.367056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:36.367510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.367947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:36.368082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:36.368595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:36.369390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:36.369390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:40.907573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:40.908303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:40.908516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:40.910160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:40.946272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:40.946787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:40.948029 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:40.949226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:40.950037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:40.950826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:40.951500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:40.952150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:40.952150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:40.952150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:40.952150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:40.952150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:40.952150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:41.339517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:41.341153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:41.341822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:41.342901 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:41.343377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:41.343625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.344653 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.346503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.347672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:41.348324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:41.349042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:41.350290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:41.350621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:41.350959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.351885 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:41.352927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.353763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:41.354383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.354929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.355335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:41.356065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:41.356930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:41.357863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:41.358120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.358706 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:41.359912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.360610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.361264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:41.361815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.362474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.362474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:41.362474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.362474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.362932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:41.363557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.364347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.364347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:41.364347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.364347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:41.364347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:41.364347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:41.364692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:41.364692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:41.365158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:41.365520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:41.365759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.366485 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.367939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:41.368738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.369114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:41.369549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:41.370082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:41.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:41.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:45.908414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:45.909537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:45.909537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:45.911205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:45.947769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:45.948032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:45.948877 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:45.949834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:45.950702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:45.951718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:45.952951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:45.953939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:45.953939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:45.953939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:45.953939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:45.953939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:45.953939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:46.341468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:46.342295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:46.343601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:46.345304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:46.345574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:46.345763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.346704 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.347996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.348449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:46.348837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.348837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:46.349092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.349653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.350285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:46.350285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.350285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:46.350285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:46.350285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:46.350285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:46.350506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:46.350977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:46.351224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.351908 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:46.353596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.354248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.355080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:46.355447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.355829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.355829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:46.355829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.355829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.357236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:46.357749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:46.358356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:46.359161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:46.359548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.360405 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:46.360916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.361504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:46.361923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:46.362430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.362813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.363376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:46.364183 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:46.364728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:46.364941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:46.365284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:46.365904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:46.366621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:46.366621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.367738 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.369344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.369936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:46.370369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.370369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:46.370682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:46.371194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:46.371738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:46.371738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:46.371738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:50.909042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:50.909758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:50.909893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:50.910829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:50.948561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:50.948957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:50.949937 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:50.950873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:50.951510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:50.952201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:50.952865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:50.953807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:50.953807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:50.953807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:50.953807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:50.953807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:50.953807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:51.342238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:51.343155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:51.343355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:51.344994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:51.345441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:51.345727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.346746 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.348865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.349631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:51.350330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:51.351129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:51.352125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:51.352580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:51.352580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.352820 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:51.354250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.355155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.355548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:51.356203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.356932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.356932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:51.356932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.356932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.357173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:51.358039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:51.359180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:51.359646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:51.359646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.359934 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:51.360938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.361695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.361952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:51.362330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.362748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.362748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:51.362748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.362748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.362920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:51.363469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.364065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.364065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:51.364065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.364065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:51.364600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:51.365018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:51.365254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:51.365508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:51.365886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:51.366261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:51.366496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.367204 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.368555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:51.369329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.369964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:51.370133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:51.371032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:51.371984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:51.371984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:55.910044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:55.910785 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:55.910993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:55.912138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:55.953391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:55.953865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:55.955033 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:55.957393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:55.958082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:55.958540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:55.958540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:55.959331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:55.959904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:55.960561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:55.960561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:55.960561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:56.343197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:56.343909 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:56.344153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:56.345596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:56.345977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:56.346252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.347282 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.348923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.349450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:56.349947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.349947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:56.350793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.351413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.352021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:56.352021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.352021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:56.352021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:56.352021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:56.352021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:56.352286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:50:56.352663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:56.352892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.353619 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:56.355096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.355798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.356570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:56.357144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.357605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.357605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:56.357605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.357605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.358731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:56.359245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:56.359787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:56.360081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:56.360302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.360897 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:50:56.361976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.362558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.363317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:56.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.364061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.364061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:56.364061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.364061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.364862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:56.365407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:50:56.365940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:50:56.366685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:56.367061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:50:56.367291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.368074 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.369101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.369608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:50:56.369933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.369933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:50:56.370221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:50:56.370812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:50:56.371361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:50:56.371361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:50:56.371361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:00.910951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:00.911758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:00.911975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:00.913722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:00.950711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:00.951337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:00.952236 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:00.953149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:00.953579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:00.954034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:00.954545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:00.955153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:00.955153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:00.955153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:00.955153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:00.955153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:00.955153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:01.344085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:01.344614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:01.344827 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:01.346423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:01.346845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:01.347159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.348242 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.349992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.350690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:01.351339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.351339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:01.351681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.352523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:01.353520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:01.353978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.354205 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:01.354946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.355842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:01.356629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.357383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:01.358064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.358842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:01.359526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:01.360765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:01.360765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:01.360765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:01.361065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:01.361065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.361640 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:01.362607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.363421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.363949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:01.364750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:01.365765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:01.366335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:01.366970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:01.367474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:01.367474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:01.367474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:01.367474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.368354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.369191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:01.369677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.370071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:01.370207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:01.370712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:01.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:01.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:05.911961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:05.913194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:05.913194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:05.913669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:05.952372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:05.952838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Traceback (most recent call last):
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:05.953804 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:05.954653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:05.955336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-74d2f408-ace2-4598-b2db-340b389b9ff8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:05.956072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:05.956843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:05.957805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:05.957805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:05.957805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:05.957805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:05.957805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:05.957805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:06.350327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:06.350327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:06.350327 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:06.351744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:06.352888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:06.352888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.353729 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.354587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.357074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.357522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:06.357921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.357921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:06.357921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:06.359515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.360172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.360942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:06.360942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.360942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:06.360942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:06.360942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:06.360942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:06.361193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:06.361422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.362152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:06.362292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:06.362454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.363508 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:06.364329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:06.366719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:06.367473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.368980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:06.369503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.369867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.369867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:06.369867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.369867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:06.369867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.370175 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:06.371836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:06.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:06.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:06.372861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:06.373342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.374592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:06.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.375372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.375372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:06.375372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.375372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:06.375372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.376590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:06.377160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:06.377809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.378219 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.379413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:06.379887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.380278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:06.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:06.382917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:06.383490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:06.383490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:10.912890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:10.913922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:10.915047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:10.918539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:10.947367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:10.948114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Traceback (most recent call last):
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:10.949152 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:10.950116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:10.950806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-113e1a91-5c14-47a3-aeb1-8e2d64eba07c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:10.951556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:10.952469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:10.953658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:10.953658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:10.953658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:10.953658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:10.953658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:10.953658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:11.350496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:11.351141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:11.351348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:11.352761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:11.353165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:11.353415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.354492 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.356463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.357111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:11.357691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.357691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:11.359803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.360643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:11.361586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:11.362058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.362473 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:11.362881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:11.364203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.365138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.365571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:11.366283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.366827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.366827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:11.366827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.366827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.367272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:11.367760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:11.368351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:11.369064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.369757 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:11.370732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:11.371206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.372238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.372614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.372989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:11.372989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.372989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:11.372989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.373755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:11.374261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.374826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.374826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:11.374826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.374826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:11.374826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:11.374826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:11.375032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:11.375291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:11.375512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.376275 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.377758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:11.378161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.378545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:11.379329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:11.379846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:11.380501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:11.380501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:15.913994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:15.915326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:15.915326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:15.918519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:15.954263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:15.954734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Traceback (most recent call last):
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:15.955774 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:15.956942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:15.957606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:15.958276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:15.959097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:15.960033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:15.960033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:15.960033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:15.960033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:15.960033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:15.960033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:16.351407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:16.352089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:16.352276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:16.354040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:16.354485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:16.354765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.355856 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.357614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.358293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:16.358873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.358873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:16.358987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.359749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.360590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:16.360590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.360590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:16.360590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:16.360913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:16.361223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:16.361429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:16.361942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:16.362213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.362786 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:16.364293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.365118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.365766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:16.366315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.366667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.366667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:16.366667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.366667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.367119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:16.367544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.368064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.368064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:16.368064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.368064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:16.368064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:16.368064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:16.368249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:16.368249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:16.368674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:16.368922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.369494 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:16.370391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.370808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.371237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.371587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.371935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:16.371935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.371935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.372445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:16.372886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:16.373409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:16.373664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:16.373900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:16.374069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.374634 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.375415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:16.375767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.376130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:16.376374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:16.376837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:16.377373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:16.377373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:20.915173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:20.916702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:20.916925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:20.918449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:20.950560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:20.950969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:20.951969 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:20.952832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:20.953291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:20.953732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:20.954244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:20.954838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:20.954838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:20.954838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:20.954838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:20.954838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:20.954838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:21.352257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:21.353054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:21.353285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:21.354261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:21.354525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:21.354742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.356110 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.358101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.358785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:21.359468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.359468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:21.360374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.361217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:21.362206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:21.362628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:21.362628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.363300 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:21.364828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.365396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.366230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:21.366600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.366989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.366989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:21.366989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.366989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.367974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:21.368494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:21.369090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:21.369897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:21.370320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.371114 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:21.372386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.372841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.373257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.373623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.374056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:21.374056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.374056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.374433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:21.374919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:21.375473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:21.375728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:21.376275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:21.376628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:21.376848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.377717 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.378904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.379287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:21.379652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.379652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:21.380160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:21.380614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:21.381197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:21.381197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:21.381197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:25.916566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:25.917536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:25.917712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:25.919357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e6c21ced-da00-494b-9211-f771092fef54 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:25.966866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:25.967358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e6c21ced-da00-494b-9211-f771092fef54 None None] Traceback (most recent call last):
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:25.969703 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6c21ced-da00-494b-9211-f771092fef54 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:25.972482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:25.973120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:25.973598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:25.973598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-e6c21ced-da00-494b-9211-f771092fef54 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:25.973890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:25.974397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:25.974907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:25.974907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:25.974907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:26.353065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:26.353740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:26.353865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:26.355140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:26.355626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:26.355813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.356643 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.357764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.358295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:26.358692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.358692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:26.358849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.359403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.360017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:26.360017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.360017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:26.361436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:26.361874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:26.362083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:26.362295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:26.362915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:26.363201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.363953 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:26.365323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.365856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.366863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:26.367237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.367593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.367593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:26.367593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.367593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.368811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:26.369298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.369792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.369792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:26.369792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.369792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:26.369792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:26.369792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:26.369995 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:26.370294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:26.370856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:26.371373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.372287 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:26.373715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.374191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.375174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.375546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.375942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:26.375942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.375942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.376170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:26.376676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:26.377225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:26.377523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:26.377523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.377800 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.378541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:26.378950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:26.379343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:26.379828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:26.380392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:26.380392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:26.380392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:30.917872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:30.918758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:30.918893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:30.920252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:30.954457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:30.954880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:30.955986 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:30.956774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:30.957696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:30.958327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:30.959074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:30.959986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:30.959986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:30.959986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:30.959986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:30.959986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:30.959986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:31.353862 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:31.354639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:31.354844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:31.355951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:31.356276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:31.356519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.357527 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.359214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.359821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:31.360447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.360447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:31.361099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.361842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:31.362705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:31.363116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.363309 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:31.364025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.364752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.365167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:31.365728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.366366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:31.367044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.367862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:31.369648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:31.369872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:31.369963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:31.370108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:31.371148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:31.371148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.371581 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:31.372331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.373087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.373602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.374213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:31.374849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:31.375584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.376438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:31.376803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:31.377688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:31.377915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:31.378315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:31.378829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:31.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:31.379475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.380405 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.382271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.382612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:31.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:31.383961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:31.384436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:31.384974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:31.384974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:31.384974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:35.918882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:35.920010 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:35.920237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:35.921605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:35.923488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:51:35.956859 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:35.957291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:35.958369 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:35.959521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:35.959905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:35.960297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:35.960297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:35.960814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:35.961454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:35.961965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:35.961965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:35.961965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:36.354926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:36.355367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:36.355492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:36.356646 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:36.356921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:36.357261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.358619 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.360908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.361723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:36.362326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.362326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:36.362825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.363582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:36.364319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:36.364578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.364614 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:36.365401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.365879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.366224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:36.366718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.367104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.367104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:36.367104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.367104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.367230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:36.367690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.368260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.368260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:36.368260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.368260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:36.369971 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:36.370951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:36.370951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:36.370951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:36.371271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:36.371501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.372231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:36.373017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.373580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.373965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.374380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.374794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:36.374794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.374794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.375017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:36.375586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.376308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.376308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:36.376308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.376308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:36.377341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:36.377645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:36.377755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:36.377916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:36.378293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:36.378516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:36.378662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.379378 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.380120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.380524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:36.380956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.380956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:36.381109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:36.381701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:36.382270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:36.382270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:36.382270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:40.920044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:40.920972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:40.920972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:40.921712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:40.959955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:40.960675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:40.961612 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:40.962877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:40.963671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:40.964375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:40.965405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:40.966511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:40.966511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:40.966511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:40.966511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:40.966511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:40.966511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:41.355642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:41.356132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:41.356328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:41.357701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:41.358134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:41.358372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.359395 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.361137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.361760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:41.362383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:41.363103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:41.363971 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:41.364352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:41.364352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:41.364569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:41.365020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:41.365315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.365830 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:41.366561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.367155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.367437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:41.367818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.368245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.368245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:41.368245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.368245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.368371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:41.368927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.369566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.369566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:41.369566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.369566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:41.371468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:41.371983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:41.372166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:41.372347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:41.372810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:41.373071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.373495 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:41.374318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.374823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.375109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.375491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.375884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:41.375884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.375884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.375980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:41.376521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.377146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.377146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:41.377146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.377146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:41.377901 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:41.378412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:41.378512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:41.379135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:41.379370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:41.379444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:41.379605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.380097 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.381319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.381716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:41.382185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.382185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:41.382500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:41.382984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:41.383532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:41.383532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:41.383532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:45.495331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:45.496617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:45.496617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:45.505556 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 02:51:45.505705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 02:51:45.505769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:51:45.540209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.540748 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.541344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.541532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Traceback (most recent call last):
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.542509 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.543872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.544542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.545285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.545285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.545285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.545285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Fixing resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:51:45.576434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, resource={'id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['97900556-2eeb-4240-89bd-ed8258b9022d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:48Z', 'updated_at': '2026-04-13T02:30:49Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9202}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.576910 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.577519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8120b9cd-5d7c-4d4d-9a57-22a6c716f312, external_ids_update=None, columns={'tag': 9202}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.577768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Traceback (most recent call last):
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.578759 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Failed to fix resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.579764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.580371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.580977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.580977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.580977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.580977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:51:45.606625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.607032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.607564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.607738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Traceback (most recent call last):
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.608672 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.610641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.611780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.612405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.612405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.612405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.612405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:51:45.644619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.644808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Traceback (most recent call last):
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.645878 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.646403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.647044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.647637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.647637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.647637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.647637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Fixing resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:51:45.728692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, resource={'id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'enable_snat': True}, 'gw_port_id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-exgz1', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'ip_address': '172.24.5.236', 'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, {'ip_address': '2001:db8::2c9', 'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 3, 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46'}, resource_type=routers, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.729073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.729416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.729570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.729570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Traceback (most recent call last):
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.730539 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.731364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Unable to update router 530b037d-a47e-4f63-bd55-041139eeee2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Failed to fix resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 13 02:51:45.732471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.733312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.734428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Fixing resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:51:45.858404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, resource={'id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'name': '', 'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'project_id': '', 'mac_address': 'fa:16:3e:92:a4:24', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000165409', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.858829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.236/24', '2001:db8::2c9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.859428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3ec69888-b9c9-4830-9d48-853dd81cf68b, lrouter_port=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.859850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Traceback (most recent call last):
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.861323 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] Maintenance task: Failed to fix resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.862835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.863675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.864349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.864349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.864349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.864349 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35d10dcf-22ce-4138-bf6e-4739bd3be1b4 None None] OVN maintenance task check_for_inconsistencies finished in 0.367 seconds
Apr 13 02:51:45.920597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:45.921031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:45.921380 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:45.922923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:45.959025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:45.959493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.960671 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.961609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.962377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:45.963067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:45.963954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:45.965111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:45.965111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:45.965111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:45.965111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:45.965111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:45.965111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:46.356474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:46.357257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:46.357438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:46.358935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:46.359494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:46.359842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.360962 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.362535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.363000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:46.363381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.363381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:46.364210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.364698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:46.365375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:46.366354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:46.367109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.367782 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:46.368559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.369618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:46.370611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.371284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:46.371696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.372301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:46.372969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:46.372969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:46.372969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:46.372969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:46.372969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:46.372969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.373281 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:46.374019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.374481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.374828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.375250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.375651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:46.375651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.375651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.376092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:46.377161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.377688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.377688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:46.377688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.377688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:46.378336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:46.379183 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:46.379411 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:46.379750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:46.380486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:46.380956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:46.381238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.382202 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.383702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.384092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:46.384412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.384412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:46.385388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:46.385845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:46.386414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:46.386414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:46.386414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:50.921312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:50.922096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:50.922232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:50.923270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:50.958547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:50.958996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:50.960022 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:50.961155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:50.961540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:50.961936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:50.961936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:50.962280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:50.962882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:50.963918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:50.963918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:50.963918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:51.357400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:51.357907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:51.358134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:51.359538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:51.360280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:51.360554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.361805 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.363659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.364057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:51.364411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.364411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:51.365499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.366054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:51.367290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:51.368174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:51.368615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.370058 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:51.370881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:51.372073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.373116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:51.373485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.373974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:51.374543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:51.374874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:51.375222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:51.375489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.376232 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:51.377350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.377871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.378553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:51.379138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.379556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.379556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:51.379556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.379556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:51.380441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.381047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.381047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:51.381047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.381047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:51.381047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:51.381047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:51.381435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:51.381801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:51.382247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:51.382470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:51.382651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.383228 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.384395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.384669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:51.384956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.384956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:51.385512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:51.385837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:51.386226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:51.386226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:51.386226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:55.922171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:55.923076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:55.923076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:55.924233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:55.959147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:55.959529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:55.960567 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:55.961306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:55.962148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:55.962723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:55.963311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:55.963899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:55.963899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:55.963899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:55.963899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:55.963899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:55.963899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:56.358211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:56.358613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:56.358746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:56.360304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:56.360823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:56.361187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.362327 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.363801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.364246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:56.364748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.364748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.364998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.365545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:56.366253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.366253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:56.366511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:56.366783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:56.366852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:56.366962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:51:56.367491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:56.367675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.368028 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:56.368544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.369056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.369310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:56.369640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.370026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:56.370430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.370932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:56.373887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:56.374519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:56.374617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:56.374742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:56.375251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:56.375534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.376079 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:51:56.376917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.377399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.377678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:56.378042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.378441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.378441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:56.378441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.378441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.378583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:56.379101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.379682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.379682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:56.379682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.379682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:51:56.380641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:51:56.381196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:51:56.381414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:51:56.381721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:51:56.382611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:56.383150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:51:56.383324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.384212 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.385820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.386244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:51:56.386625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.386625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:51:56.387442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:51:56.387958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:51:56.388519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:51:56.388519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:51:56.388519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:00.922996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:00.924419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:00.924419 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:00.925878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:00.962642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:00.963082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:00.964380 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:00.965359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:00.965786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:00.966127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:00.966127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:00.966421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:00.966846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:00.967346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:00.967346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:00.967346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:01.359152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:01.359456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:01.359593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:01.361109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:01.361538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:01.361793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.362809 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.364391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.364969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:01.365593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.365593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.365989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.366964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:01.367746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.367746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:01.367746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:01.367746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:01.367746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:01.367746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:01.368078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:01.368078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.368671 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:01.369639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.370228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.370655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:01.370963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.371338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.371338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:01.371338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.371338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.371702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:01.372176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.372720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.372720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:01.372720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.372720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:01.374297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:01.374966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:01.375247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:01.375470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:01.375905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:01.376161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.376767 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:01.378348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.379046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.379760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:01.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.380571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.380571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:01.380571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.380571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.381666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:01.382157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.382766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.382766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:01.382766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.382766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:01.382766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:01.382766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:01.382962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:01.383265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:01.383681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:01.383956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:01.384186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.384854 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.386433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.386817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:01.387211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.387211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:01.388027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:01.388760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:01.389263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:01.389263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:01.389263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:05.923932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:05.924797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:05.925061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:05.926396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:05.967341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:05.967930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:05.968892 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:05.969953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:05.970632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:05.971321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:05.972183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:05.972709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:05.972709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:05.972709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:05.972709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:05.972709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:05.972709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:06.360113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:06.360530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:06.360624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:06.362176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:06.362624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:06.362910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.363904 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.365655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.366257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.366836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:06.367569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:06.368500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:06.368925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:06.368989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.369821 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:06.370702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.371549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:06.372544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.373168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:06.373625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.374197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:06.374792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:06.375617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:06.375952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:06.376104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:06.376792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:06.377030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.377741 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:06.379274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.380144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.380901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:06.381275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.381675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.381675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:06.381675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.381675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.382776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:06.383302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:06.383972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:06.384606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:06.384961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:06.385266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.385978 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.386750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.387150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:06.387515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:06.387957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:06.388551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:06.388551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:06.388551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:06.388551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:06.388551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:06.388551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:10.925101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:10.925928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:10.926248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:10.927983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:10.971056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:10.971056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Traceback (most recent call last):
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:10.971444 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:10.972477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:10.973486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7f233efb-6580-4fa2-821d-5fbf6bc94f7e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:10.973913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:10.974452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:10.975095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:10.975095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:10.975095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:10.975095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:10.975095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:10.975095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:11.360958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:11.361928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:11.361928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:11.363255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:11.363534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:11.364191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.365446 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.367833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:11.368083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.368083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.369117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.369504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:11.369877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.369877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:11.370707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:11.370707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:11.370707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:11.370707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:11.371248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:11.371484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:11.375789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.376334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.376700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.376972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.377236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.377549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:11.378252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:11.379175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:11.379386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:11.379640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:11.380213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:11.380482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.381210 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:11.382747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.383074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.383752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:11.384177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.384566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.384566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:11.384566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.384566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.386084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:11.389578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:11.390166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:11.390484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:11.390484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:11.390484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.390887 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.392118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.392400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:11.392643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.392643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:11.393459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:11.395110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:11.395480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:11.395480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:11.395480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:15.925935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:15.927627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:15.927627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:15.929302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:15.959888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:15.960102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Traceback (most recent call last):
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:15.960582 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:15.961316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:15.961823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:15.962562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:15.963390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:15.964454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:15.964454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:15.964454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:15.964454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:15.964454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:15.964454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:16.362144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:16.362553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:16.362696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:16.364228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:16.364670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:16.364887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.365962 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.367515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.368180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.368794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:16.369575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.370520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.370520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:16.370520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.370520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:16.370520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:16.370806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:16.370806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:16.370806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:16.371393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:16.371713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.372528 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:16.373351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.373942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.374243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:16.374700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.375112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.375112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:16.375112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.375112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.375239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:16.375741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.376347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.376347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:16.376347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.376347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:16.378839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:16.379626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:16.379823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:16.380117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:16.380718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:16.381108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.381935 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:16.383252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.383754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.384141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:16.384762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.385252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.385252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:16.385252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.385252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.385391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:16.386150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.386775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.386775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:16.386775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.386775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:16.386775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:16.387110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:16.387285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:16.387555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:16.388143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:16.388480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:16.388811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.389720 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.391191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.391630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:16.392038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.392038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:16.392860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:16.393400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:16.394034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:16.394034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:16.394034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:20.926893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:20.928368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:20.928368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:20.930477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:20.966042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:20.966621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Traceback (most recent call last):
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:20.967821 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:20.968981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:20.969705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:20.970437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:20.971365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:20.972258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:20.972258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:20.972258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:20.972258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:20.972258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:20.972258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:21.362867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:21.363216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:21.363216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:21.364287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:21.364568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:21.364726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.365393 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.366521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.367188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.367795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:21.368623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.369630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.369630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:21.369630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.369630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:21.370729 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:21.370887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:21.370966 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:21.371054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:21.371449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:21.371653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.372148 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:21.372859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.373775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:21.374587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.375231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:21.375842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.376908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:21.379200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:21.379750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:21.379897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:21.380094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:21.380624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:21.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.381708 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:21.382606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.382949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.383487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:21.383809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.384089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.384089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:21.384089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.384089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.384560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:21.384917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.385289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.385289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:21.385289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.385289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:21.387219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:21.387604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:21.387775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:21.388056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:21.388525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:21.388878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:21.389157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.389904 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.391319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.391707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:21.392093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.392093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:21.392814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:21.393352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:21.393929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:21.393929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:21.393929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:25.927732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:25.928385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:25.928555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:25.929599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:25.968828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:25.969318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] Traceback (most recent call last):
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:25.970359 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:25.971688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:25.972515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:25.973371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:25.973902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:25.974573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:25.974573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:25.974573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:25.974573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:25.974573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:25.974573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:26.363683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:26.364767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:26.365020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:26.365965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:26.366248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:26.366415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.367233 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.368965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.369571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:26.370445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.370445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.370785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.371716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:26.372621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.373095 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:26.374203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.374827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.375296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:26.375687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.376218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.376218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:26.376218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.376218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.376708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:26.377264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.377880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.377880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:26.377880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.377880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:26.379545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:26.379992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:26.380233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:26.380456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:26.380815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:26.381076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.381639 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:26.383031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.383520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.383863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:26.384269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.384727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.384727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:26.384727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.384727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.384935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:26.385459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.385973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.385973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:26.385973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.385973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:26.387738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:26.388207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:26.388397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:26.388674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:26.389029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:26.389456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:26.389605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.390234 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.391699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.392103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:26.392520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.392520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:26.393447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:26.393947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:26.394567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:26.394567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:26.394567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:30.928712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:30.930324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:30.930324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:30.931533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:30.966268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:30.966537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Traceback (most recent call last):
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:30.967362 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:30.968190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:30.969185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:30.969905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:30.970839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:30.972074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:30.972074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:30.972074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:30.972074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:30.972074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:30.972074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:31.364713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:31.365608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:31.365833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:31.367495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:31.367956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:31.368431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.369414 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.370635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.371528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.372430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:31.372926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:31.373808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.374726 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:31.375133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.375946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:31.376889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.377606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:31.377961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.378657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:31.379948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:31.380298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:31.380464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:31.380668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:31.381078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:31.381736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.382176 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:31.383426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.383785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.384464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:31.384752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.385045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.385045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:31.385045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.385045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.385616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:31.386017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.386420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.386420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:31.386420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.386420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:31.388118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:31.388533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:31.388673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:31.388903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:31.389321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:31.389541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:31.389691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.390299 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.391472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.392077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:31.392549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:31.393046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:31.393628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:31.393628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:31.393628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:31.393628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:31.393628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:31.393628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:35.930339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:35.931351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:35.931351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:35.933086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:35.934993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=434 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:52:35.971245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:35.971820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Traceback (most recent call last):
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:35.973152 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:35.974531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:35.975780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:35.976832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:35.977819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:35.978371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:35.978371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:35.978371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:35.978371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:35.978371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:35.978371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:36.365624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:36.366348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:36.366580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:36.367660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:36.367989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:36.368213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.369141 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.370292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.370728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:36.371133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.371133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.371636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.372617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:36.373542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.373542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:36.373542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:36.375230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:36.375230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:36.375230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:36.376136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:36.376382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.378044 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:36.378871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.379463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.379808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:36.380251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.380641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.380641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:36.380641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.380641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.381917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:36.388262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:36.388912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:36.390479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.390883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:36.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:36.391855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:36.392509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:36.392854 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.393186 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.393186 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:36.393186 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.393186 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.394285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.394602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:36.394937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.394937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:36.397638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:36.398309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:36.398691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:36.398691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:36.398691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:40.930869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:40.932526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:40.932526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:40.933945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:40.967039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:40.967300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Traceback (most recent call last):
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:40.968206 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:40.969034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:40.969720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:40.970409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:40.971231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:40.972180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:40.972180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:40.972180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:40.972180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:40.972180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:40.972180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:41.366331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:41.367207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:41.367350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:41.368857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:41.369316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:41.369579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.370547 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.372214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.372919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:41.373736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.373736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.373852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.374401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:41.374904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.374904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:41.374904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:41.374904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:41.375130 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:41.375285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:41.375800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:41.376022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.376579 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:41.378177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.378627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.379777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:41.380163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.380516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.380516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:41.380516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.380516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.381982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:41.382473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.383055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.383055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:41.383055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.383055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:41.383055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:41.383055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:41.383251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:41.383408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:41.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:41.384108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.384731 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:41.386223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.386681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.387653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:41.388041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.388427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.388427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:41.388427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.388427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.389248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:41.389813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.390472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.390472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:41.390472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.390472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:41.390472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:41.390472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:41.390651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:41.390796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:41.391802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:41.392186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:41.392416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.393042 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.393885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.394275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:41.394634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.394634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:41.394839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:41.395290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:41.395769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:41.395769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:41.395769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:45.931912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:45.932961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:45.932961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:45.934718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:45.975238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:45.975611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Traceback (most recent call last):
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:45.976689 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:45.977721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:45.978497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:45.979465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:45.980304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:45.980928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:45.980928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:45.980928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:45.980928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:45.980928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:45.980928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:46.367279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:46.367994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:46.368186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:46.369591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:46.369940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:46.370198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.371098 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.372855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.373519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.374126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:46.374865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:46.375837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:46.376283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:46.376501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.377217 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:46.378749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.379369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.379919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:46.380317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.380696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.380696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:46.380696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.380696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.381214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:46.381706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.382295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.382295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:46.382295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.382295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:46.382720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:46.383082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:46.383241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:46.383417 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:46.383835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:46.384115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.384681 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:46.385679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.386161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.386773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:46.387182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.387573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.387573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:46.387573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.387573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.388129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:46.388594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.389133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.389133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:46.389133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.389133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:46.390532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:46.390806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:46.390939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:46.391233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:46.391822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:46.392198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:46.392421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.393192 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.394143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.394521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:46.394884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.394884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:46.395394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:46.395858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:46.396403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:46.396403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:46.396403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:50.932983 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:50.934482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:50.934482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:50.935998 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:50.972039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:50.972256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Traceback (most recent call last):
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:50.972871 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:50.973711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:50.974158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:50.974623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:50.974623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-51dbeb4a-f24a-48bf-9755-3cabe3880e6d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:50.974703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:50.975232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:50.975919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:50.975919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:50.975919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:51.368277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:51.368945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:51.369100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:51.370676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:51.371095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:51.371395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.372423 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.373908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.374516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.375108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:51.375772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:51.376710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:51.377228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:51.377591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.378260 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:51.379517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.380154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.380952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:51.381398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.381804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.381804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:51.381804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.381804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.382031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:51.382568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.383242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.383242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:51.383242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.383242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:51.383242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:51.383648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:51.383848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:51.384144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:51.384569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:51.384795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.385507 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:51.386908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.387409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.388556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:51.388959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.389397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.389397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:51.389397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.389397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.389949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:51.390507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.391112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.391112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:51.391112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.391112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:51.391112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:51.391112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:51.391350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:51.391390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:51.391734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:51.392037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:51.392200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.392786 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.393711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.394184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:51.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:51.394702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:51.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:51.395732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:51.395732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:51.395732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:55.933780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:55.934757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:55.934911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:55.936728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:55.970724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:55.971341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:55.972420 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:55.973574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:55.974249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:55.974890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:55.974890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:55.975043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:55.975905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:55.976860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:55.976860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:55.976860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:56.369131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:56.369868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:56.370067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:56.371583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:56.372051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:56.372295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.373289 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.375243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.375992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:56.376731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.376731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.376955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.378140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:52:56.379223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:56.379755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.379833 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:56.380834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.381732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:56.382291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.382702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:56.383198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.383866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:56.384507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:56.384507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:56.384507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:56.384772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:56.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:56.385565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.386179 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:52:56.387340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.387819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.388320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:56.388823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.389347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.389347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:56.389347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.389347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.389795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:56.390340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.390880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.390880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:56.390880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.390880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:52:56.391226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:52:56.391528 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:52:56.391696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:52:56.391878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:52:56.392234 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:56.392468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:52:56.392621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.393271 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.394163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.394623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:52:56.395108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.395108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:52:56.395198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:52:56.395673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:52:56.396326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:52:56.396326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:52:56.396326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:00.934555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:00.935860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:00.935860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:00.936888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:00.975355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:00.975633 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:00.976065 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:00.976951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:00.977897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:00.978744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:00.980348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:00.981594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:00.981594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:00.981594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:00.981594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:00.981594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:00.981594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:01.370060 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:01.370569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:01.370669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:01.371733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:01.372050 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:01.372311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.373628 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.375591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.376272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:01.376888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.376888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.377854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:01.379529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.380062 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:01.380592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.381130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:01.381889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.382310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.382310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:01.382310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.382310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.382545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:01.383054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.383715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.383715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:01.383715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.383715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:01.384275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:01.385172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:01.385427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:01.385678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:01.386141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:01.386365 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.387027 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:01.388501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.388951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.390091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:01.390451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.390876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.390876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:01.390876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.390876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.391944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:01.392478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:01.393027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:01.393304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:01.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:01.393690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.394134 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.394930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.395309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:01.395686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.395686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:01.395973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:01.396498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:01.397081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:01.397081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:01.397081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:05.935376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:05.936453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:05.936453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:05.937703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:05.971850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:05.972186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:05.973195 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:05.974114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:05.974797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:05.975519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:05.976375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:05.977474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:05.977474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:05.977474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:05.977474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:05.977474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:05.977474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:06.370996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:06.371658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:06.371752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:06.373194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:06.373632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:06.373945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.375117 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.376401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.376994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.377609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:06.378350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:06.379242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:06.379598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:06.379865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:06.380125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.380735 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:06.382241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.382756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:06.383992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.384437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.384437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:06.384437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.384437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.385844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:06.386461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:06.387071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:06.387371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:06.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.388344 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:06.389860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.390360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.391336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:06.391684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.392094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.392094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:06.392094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.392094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.393223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:06.393741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:06.394316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:06.394726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:06.395159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:06.395479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.396194 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.397407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:06.397791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.397791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:06.397970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:06.398470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:06.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:06.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:06.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:10.936181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:10.939283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:10.939602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:10.941134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-defdf0f7-c601-410e-998d-5b34648c39b4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:10.982436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:10.982757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-defdf0f7-c601-410e-998d-5b34648c39b4 None None] Traceback (most recent call last):
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:10.985370 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-defdf0f7-c601-410e-998d-5b34648c39b4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:10.987476 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:10.988208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:10.988868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:10.988868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-defdf0f7-c601-410e-998d-5b34648c39b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:10.989145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:10.990198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:10.991261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:10.991261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:10.991261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:11.372257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:11.372838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:11.373122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:11.375164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:11.375581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:11.375851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.377026 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.378159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.378887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.379612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:11.380426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:11.381835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:11.382276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:11.382342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:11.382533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.383101 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:11.384648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.385559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.386019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:11.386542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.386949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.386949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:11.386949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.386949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.391850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:11.392761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:11.393394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:11.393648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:11.393847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.394314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:11.394960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:11.395516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:11.395759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.396050 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:11.397019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.397480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.397958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.398347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.398741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:11.398741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.398741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.399426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:11.399990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:11.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.400789 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.401781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.402178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:11.402567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.402567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:11.402920 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:11.403399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:11.403918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:11.403918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:11.403918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:15.937918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:15.938760 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:15.938900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:15.940525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ab0fe1fc-377b-4deb-8899-cdd6b1887e56 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:15.976946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:15.977398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ab0fe1fc-377b-4deb-8899-cdd6b1887e56 None None] Traceback (most recent call last):
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:15.978358 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ab0fe1fc-377b-4deb-8899-cdd6b1887e56 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:15.979308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:15.979794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ab0fe1fc-377b-4deb-8899-cdd6b1887e56 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:15.980288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:15.980998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:15.981993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:15.981993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:15.981993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:15.981993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:15.981993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:15.981993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:16.372991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:16.373713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:16.373891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:16.374989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:16.375314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:16.375537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.376358 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.377763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.378373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:16.378989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.378989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.379646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.380689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:16.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:16.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:16.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:16.381488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:16.381766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:16.382254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:16.382727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.383612 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:16.385239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.385852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.386981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:16.387379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.387768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.387768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:16.387768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.387768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:16.388867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.389379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.389894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:16.392535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:16.392897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:16.392988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:16.393127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:16.393667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:16.394081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.394891 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:16.395862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:16.396388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.397358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.398143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:16.398143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.398143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.398984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:16.399496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:16.399992 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:16.400458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:16.400895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:16.401183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.402078 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.403867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.404248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:16.405313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.405313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:16.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:16.406322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:16.406873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:16.406873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:16.406873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:20.938889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:20.940338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:20.940453 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:20.941586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cb3179c4-4bf8-465e-a9eb-4345e2bfc0b5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:20.979445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:20.979652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cb3179c4-4bf8-465e-a9eb-4345e2bfc0b5 None None] Traceback (most recent call last):
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:20.980395 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cb3179c4-4bf8-465e-a9eb-4345e2bfc0b5 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:20.981449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:20.982163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cb3179c4-4bf8-465e-a9eb-4345e2bfc0b5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:20.982625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:20.983149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:20.983717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:20.983717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:20.983717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:20.983717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:20.983717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:20.983717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:21.373888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:21.374348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:21.374470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:21.375219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:21.375437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:21.375601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.376242 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.377177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.377556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:21.377916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.377916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.378375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.378872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:21.379458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.379458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:21.381387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:21.382064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:21.382204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:21.382347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:21.382782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:21.382980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.383582 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:21.384421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.385019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.385291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:21.385645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.386071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.386071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:21.386071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.386071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:21.386267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.386726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:21.393209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:21.393557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:21.393660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:21.393915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:21.394422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:21.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.395910 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:21.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.397980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.398639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.399250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:21.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:21.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.400525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:21.401303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:21.402308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:21.402668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:21.402668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.403119 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.404725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.405341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:21.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:21.406441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:21.407112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:21.407575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:21.407575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:21.407575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:25.940352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:25.941062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:25.941175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:25.942526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:25.981134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:25.981388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:25.982117 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:25.982964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:25.983445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:25.983927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:25.984506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:25.985225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:25.985225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:25.985225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:25.985225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:25.985225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:25.985225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:26.374499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:26.375136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:26.375288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:26.376856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:26.376856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:26.376856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.377681 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.378800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.379355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.380224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:26.380984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.382087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.382087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:26.382087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.382087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:26.382087 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:26.382371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:26.382371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:26.382505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:26.382916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:26.383126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.383601 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:26.384584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:26.385422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.386201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.386551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:26.386802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.387196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.387661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:26.393683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:26.394185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:26.394363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:26.394617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:26.395151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:26.395427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.396229 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:26.397886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.399144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.400682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.401563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.402448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:26.402448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.402448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.402862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:26.403524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:26.404236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:26.404541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:26.404764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:26.405016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.405759 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.406483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:26.406879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.407331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:26.407435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:26.407800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:26.408163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:26.408163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:30.941071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:30.941736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:30.941736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:30.942959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:30.981695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:30.982294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:30.983446 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:30.984344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:30.984954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:30.985548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:30.986285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:30.987170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:30.987170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:30.987170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:30.987170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:30.987170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:30.987170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:31.375344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:31.376025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:31.376206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:31.377219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:31.377508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:31.377741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.378587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.380096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.380730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.381334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:31.382112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.382967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.382967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:31.382967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.382967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:31.382967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:31.383295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:31.383295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:31.383295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:31.383652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:31.383862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.384386 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:31.385094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.385653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.385907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:31.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.386661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.386661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:31.386661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.386661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:31.386865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.387349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.387851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:31.394212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:31.394981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:31.394981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:31.394981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:31.395636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:31.396035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.396787 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:31.397698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.398544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:31.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:31.399967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.400639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.401548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:31.403504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:31.404111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:31.404311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:31.404596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:31.405145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:31.405522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:31.405771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.406644 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.408311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.408662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:31.409017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.409017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:31.409909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:31.410497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:31.411125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:31.411125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:31.411125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:35.941911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:35.942638 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:35.942713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:35.944156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-139f691e-3fa7-4983-ab07-fad104888bd2 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:35.947270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:53:35.992424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:35.992734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-139f691e-3fa7-4983-ab07-fad104888bd2 None None] Traceback (most recent call last):
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:35.993621 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-139f691e-3fa7-4983-ab07-fad104888bd2 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:35.994657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:35.995350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-139f691e-3fa7-4983-ab07-fad104888bd2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:35.996028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:35.996821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:35.997816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:35.997816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:35.997816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:35.997816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:35.997816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:35.997816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:36.376136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:36.376772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:36.376887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:36.377702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:36.377907 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:36.378085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.378697 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.379536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.380215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.380761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:36.381533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.382401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.382401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:36.382401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.382401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:36.383166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:36.383898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:36.384273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:36.384366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:36.384943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:36.385602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.385890 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:36.387301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.387840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.388941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:36.389846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:36.390560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.391281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.392220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:36.394684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:36.395195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:36.395260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:36.395413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:36.396137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:36.396559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.397536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:36.398503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.399053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.399390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.399782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.400219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:36.400219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.400219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.400540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:36.401121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.401605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.401605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:36.401605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.401605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:36.403927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:36.404396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:36.404582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:36.404905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:36.405546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:36.405974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:36.406257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.407016 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.408372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.408773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:36.409172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.409172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:36.409973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:36.410488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:36.411036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:36.411036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:36.411036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:40.942618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:40.943482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:40.943723 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:40.945222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:40.990259 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:40.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] Traceback (most recent call last):
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:40.992375 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:40.993624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:40.994077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:40.994622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:40.994622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-e23c0bd6-5775-4fda-8efd-a4f9d6431ea4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:40.994781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:40.995358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:40.995995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:40.995995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:40.995995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:41.376954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:41.377819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:41.378214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:41.379235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:41.379580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:41.379824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.380634 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.382160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.382861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.383470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:41.384259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:41.385187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:41.385652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:41.385652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.385787 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:41.386834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.387423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.387704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:41.388081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.388446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.388446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:41.388446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.388446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:41.388710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.389166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.389635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:41.395056 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:41.395326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:41.395412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:41.395511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:41.395981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:41.396362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.397104 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:41.397956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.398452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.399389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.399768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.400225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:41.400225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.400225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.401493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:41.401965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.402483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.402483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:41.402483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.402483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:41.404451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:41.404816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:41.404930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:41.405120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:41.405609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:41.405961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:41.406259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.407079 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.407979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.408343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:41.408676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.408676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:41.409105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:41.409549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:41.410100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:41.410100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:41.410100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:45.943526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:45.944804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:45.945081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:45.946443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:45.984057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:45.984887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:45.986120 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:45.988025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:45.988822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:45.989594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:45.989594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:45.990406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:45.990993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:45.991675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:45.991675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:45.991675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:46.377951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:46.378786 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:46.378990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:46.380413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:46.380861 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:46.381173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.382174 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.383840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.384534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:46.385127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.385127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:46.385783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.386669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:46.387546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:46.387973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.388566 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:46.389336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:46.390140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.390841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.391475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:46.391998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.392851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.393381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:46.395484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:46.396098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:46.396204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:46.396357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:46.396852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:46.397169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.397733 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:46.398699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.399225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.399774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.400221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.400625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:46.400625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.400625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.401725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:46.402256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.402736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.402736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:46.402736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.402736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:46.404929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:46.405293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:46.405404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:46.405575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:46.406112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:46.406421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:46.406604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.407398 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.408613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.409550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:46.410186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:46.410976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:46.411975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:46.411975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:46.411975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:46.411975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:46.411975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:46.411975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:50.944599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:50.945564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:50.945757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:50.947418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:50.984923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:50.985389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:50.986577 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:50.987736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:50.988254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:50.988625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:50.988625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:50.988899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:50.989949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:50.990508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:50.990508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:50.990508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:51.378906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:51.379688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:51.379842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:51.381008 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:51.381385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:51.381556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.382341 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.383638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.384463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:51.385054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:51.385763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:51.386617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:51.387220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:51.387368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:51.387538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:51.388024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:51.388277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.388818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:51.390306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.391351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.391832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:51.392254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.392605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.392605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:51.392605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.392605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.393520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:51.394058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.394595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.394595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:51.394595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.394595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:51.396124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:51.396584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:51.396765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:51.396984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:51.397787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:51.398213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.398879 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:51.400172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.400547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.401119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.401751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:51.401751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.401751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.402349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:51.402768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.403291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.403291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:51.403291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.403291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:51.405382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:51.405803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:51.406068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:51.406334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:51.406842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:51.407253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:51.407494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.408350 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.410268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:51.410516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.410516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:51.411267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:51.411589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:51.411964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:51.411964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:51.411964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:55.945537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:55.946302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:55.946523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:55.947854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:55.982683 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:55.983195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:55.984569 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:55.985898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:55.986684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:55.987345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:55.988098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:55.988962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:55.988962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:55.988962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:55.988962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:55.988962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:55.988962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:56.379841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:56.381074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:56.381074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:56.382407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:56.383318 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:56.383527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.384366 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.386049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.386402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:56.386787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.386787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:56.387757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.388213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.388751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:56.388751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.388751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:56.388751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:56.389696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:56.389696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:56.389696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:53:56.390290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:56.390632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.391545 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:56.392489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.392944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.393777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:56.394201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.394652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.394652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:56.394652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.394652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.395461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:56.395994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.396637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.396637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:56.396637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.396637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:56.396637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:56.396952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:56.397135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:56.397336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:56.398137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:56.398517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.399416 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:53:56.400865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.401336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.402428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.402814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.403295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:56.403295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.403295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.404379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:56.404896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.405470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.405470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:56.405470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.405470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:53:56.405735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:53:56.406622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:53:56.406854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:53:56.407237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:53:56.407837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:56.408236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:53:56.408453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.409344 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.410959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.411333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:53:56.411669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.411669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:53:56.412703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:53:56.413131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:53:56.413622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:53:56.413622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:53:56.413622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:00.946468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:00.947241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:00.947241 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:00.948682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:00.987566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:00.988073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:00.989183 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:00.990401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:00.991097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:00.991770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:00.992592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:00.993593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:00.993593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:00.993593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:00.993593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:00.993593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:00.993593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:01.380776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:01.381842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:01.382153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:01.383744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:01.384225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:01.384497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.385634 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.387485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.388408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:01.389110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.389110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:01.389455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.390366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:01.391413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:01.391884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.392911 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:01.394408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.394927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.395793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:01.396183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.396550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.396550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:01.396550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.396550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.397589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:01.398092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:01.398626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:01.398838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:01.399391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:01.399776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.400733 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:01.402153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.402626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.403617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.404047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.404450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:01.404450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.404450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.405368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:01.405899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.406442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.406442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:01.406442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.406442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:01.406442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:01.407049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:01.407290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:01.407637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:01.408277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:01.409169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:01.409443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.410516 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.412276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.412660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:01.413097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.413097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:01.414074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:01.414578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:01.415159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:01.415159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:01.415159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:05.947298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:05.948296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:05.948296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:05.949458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:05.989652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:05.990195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:05.991123 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:05.992178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:05.992905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:05.993746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:05.994690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:05.995762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:05.995762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:05.995762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:05.995762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:05.995762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:05.995762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:06.381837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:06.382846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:06.383109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:06.384589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:06.384965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:06.385254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.386277 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.388132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.388774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:06.389387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.389387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:06.389872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.390695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:06.391639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:06.392093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:06.392406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.393160 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:06.394695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.395322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.396086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:06.396503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.396924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.396924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:06.396924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.396924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.397740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:06.398299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:06.399108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:06.399678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:06.400099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.401116 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:06.402438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.402972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.403895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.404352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.404741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:06.404741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.404741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.405683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:06.406240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.406804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.406804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:06.406804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.406804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:06.406804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:06.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:06.407797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:06.408175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:06.409354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:06.409817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:06.410163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.411328 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.412927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.413388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:06.413772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.413772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:06.414732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:06.415250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:06.415802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:06.415802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:06.415802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:10.948072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:10.948945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:10.949186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:10.950585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:10.989535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:10.990087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:10.991238 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:10.992547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:10.993299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:10.994277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:10.995129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:10.996350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:10.996350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:10.996350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:10.996350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:10.996350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:10.996350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:11.382737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:11.383414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:11.383572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:11.384726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:11.384915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:11.385173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.386054 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.387272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.387924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:11.388599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.388599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.388948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.389855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:11.390781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.390781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:11.391078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:11.391441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:11.391569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:11.391741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:11.392395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:11.392660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.393250 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:11.394584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.395122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.396084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:11.396452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.396833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.396833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:11.396833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.396833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.397940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:11.398481 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:11.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.399054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:11.399187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:11.400832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:11.400832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:11.400832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:11.400832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:11.401055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.401774 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:11.403196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.403646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.404644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:11.404987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.405415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.405415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:11.405415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.405415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.406543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:11.407068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.407557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.407557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:11.407557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.407557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:11.407557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:11.407714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:11.407758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:11.407928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:11.408460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:11.408822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:11.409090 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.409833 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.410772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.411148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:11.411477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.411477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:11.411861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:11.412341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:11.412816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:11.412816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:11.412816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:15.949036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:15.950394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:15.950627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:15.952076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:15.987073 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:15.987437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:15.988229 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:15.988915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:15.989834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:15.990556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:15.991282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:15.992200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:15.992200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:15.992200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:15.992200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:15.992200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:15.992200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:16.383526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:16.384325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:16.384482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:16.385641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:16.385962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:16.386315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.387135 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.387984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.388776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.389550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:16.390437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.391533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.391533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:16.391533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.391533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:16.391817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:16.392263 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:16.392374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:16.392615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:16.393247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:16.393685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.394273 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:16.395773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.396329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.397335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:16.398103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.398488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.398488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:16.398488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.398488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.398896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:16.399504 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.400195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.400195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:16.400195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.400195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:16.400195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:16.400382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:16.400445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:16.400688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:16.401202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:16.401428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.402286 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:16.403568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.404044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.405174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:16.405530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.406000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.406000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:16.406000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.406000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.408125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:16.408799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:16.409414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:16.409671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:16.410358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:16.410709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:16.410869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.411523 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.412978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.413418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:16.413800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.413800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:16.414990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:16.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:16.416065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:16.416065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:16.416065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:20.950325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:20.951062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:20.951186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:20.952288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:20.992134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:20.992450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:20.993268 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:20.994372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:20.995176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:20.995946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:20.996812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:20.997785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:20.997785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:20.997785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:20.997785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:20.997785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:20.997785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:21.384474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:21.385259 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:21.385439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:21.388246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:21.388246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:21.388246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.389416 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.391435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.392214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:21.392818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.392818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.393539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.394398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:21.395242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.395676 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:21.396457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.397152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.397417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:21.397767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.398142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.398142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:21.398142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.398142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.398448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:21.398971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.399568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.399568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:21.399568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.399568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:21.400233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:21.400732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:21.400856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:21.401023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:21.401742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:21.402135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.402858 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:21.403842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.404359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.404763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:21.405177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.405578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.405578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:21.405578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.405578 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.405835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:21.406367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.406897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.406897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:21.406897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.406897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:21.409093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:21.409860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:21.410113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:21.410428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:21.410989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:21.411407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:21.411638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.412628 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.414297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.414670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:21.415075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.415075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:21.415968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:21.416465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:21.416981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:21.416981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:21.416981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:25.951155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:25.952120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:25.952120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:25.952927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:25.994889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:25.995215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:25.996102 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:25.997094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:25.997692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:25.998310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:25.999173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.000126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:26.000126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.000126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.000126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:26.000126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.000126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:26.385594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:26.386324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:26.386514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:26.387906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:26.388485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:26.388741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.389763 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.391530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.392196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:26.392825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.392825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.393182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.393964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:26.395049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.395049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:26.395049 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:26.395225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:26.395225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:26.395401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:26.395940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:26.396189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.396871 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:26.398644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.399492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.399877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:26.400781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.401511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:26.402202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:26.403163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:26.403735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:26.403735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:26.403735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:26.403735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.404043 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:26.405042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.405468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.405937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:26.406297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.406671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.406671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:26.406671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.406671 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.407111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:26.407585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.408094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.408094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:26.408094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.408094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:26.409507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:26.409882 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:26.410021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:26.410253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:26.412037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:26.412037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:26.412037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.412228 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.413199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.413570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:26.414067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.414067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:26.414206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:26.414675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:26.415210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:26.415210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:26.415210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:30.951866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:30.952752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:30.952752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:30.953354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:30.990860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:30.991372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Traceback (most recent call last):
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:30.992767 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:30.993770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:30.994485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:30.995174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:30.996062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:30.996916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:30.996916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:30.996916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:30.996916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:30.996916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:30.996916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:31.386481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:31.387398 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:31.387615 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:31.389267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:31.389659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:31.390177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.391280 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.393119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.393503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:31.393878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.393878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.395022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:31.396000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.396000 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:31.396000 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:31.396116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:31.396251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:31.396409 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:31.396953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:31.397231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.397841 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:31.399247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.399772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.400609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:31.400987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.401374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.401374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:31.401374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.401374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.402332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:31.402804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.403377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.403377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:31.403377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.403377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:31.403377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:31.403377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:31.403584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:31.403734 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:31.404445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:31.404819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.405743 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:31.407592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.408524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.409170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:31.409788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.410542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.410542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:31.410542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.410542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.411027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:31.411542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:31.412082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:31.412559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:31.412915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:31.413191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.414138 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.415081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.415473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:31.415832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.415832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:31.416223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:31.416689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:31.417305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:31.417305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:31.417305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:35.952922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:35.953967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:35.953967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:35.955168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:35.957462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:54:36.007840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.008226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Traceback (most recent call last):
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.009105 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.010296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.010939 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d2e45983-cfad-45f8-a730-15f4f4e4a887 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:36.011598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:36.012439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.013425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:36.013425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.013425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.013425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:36.013425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.013425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:36.387609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:36.388479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:36.388628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:36.389925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.390141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.390347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.391035 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.392783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.395506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:36.396485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.397221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.397221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:36.397221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.397221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:36.397221 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:36.397718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:36.397865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:36.398020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:36.398483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.398664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.399198 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:36.400581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.401180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.401860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:36.402273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.402675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.402675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:36.402675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.402675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.403590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:36.404136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.404707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.404707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:36.404707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.404707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:36.404707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:36.404908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:36.405294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:36.405513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:36.405934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.406130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.406612 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:36.407719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.408159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.408683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:36.409019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.409393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.409393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:36.409393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.409393 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.409826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:36.410313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.410972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.410972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:36.410972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.410972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:36.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:36.412340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:36.412561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:36.412871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:36.413249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.413413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:36.413526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.413988 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.414707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.415135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:36.415492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:36.415907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:36.416441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:36.416441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:36.416441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:36.416441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:36.416441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:36.416441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:40.953539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:40.954385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:40.954586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:40.955903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:41.261782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.262595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.263321 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task check_fdb_aging_settings failed after 0.309 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.264261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.264972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:41.265644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:41.266529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.267516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:41.267516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.267516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.267516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:41.267516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.267516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:41.388593 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:41.389137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:41.389203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:41.390180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.390482 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.390634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.391356 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.392974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.393391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:41.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.393670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.394585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.395165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:41.395823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.395823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:41.397603 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:41.398134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:41.398231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:41.398618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:41.398941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.399063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.399571 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:41.400541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.401372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:41.402121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.402795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:41.403495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.404614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:41.405663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:41.406167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:41.406444 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:41.406686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:41.407187 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.407429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.408068 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:41.409569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.410071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.411175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:41.411560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.411992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.411992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:41.411992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.411992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.413375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:41.413984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:41.414584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:41.414924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.415198 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:41.415499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.416178 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.417024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.417399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:41.417679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.417679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:41.418100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:41.418518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:41.418948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:41.418948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:41.418948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:45.954464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:45.955542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:45.955542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:45.956823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:45.993988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:45.994319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:45.995498 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:45.996659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:45.997388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:45.998102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:45.998899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:45.999860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:45.999860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:45.999860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:45.999860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:45.999860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:45.999860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:46.389511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:46.390132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:46.390261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:46.391550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:46.391831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:46.392084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.392733 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.394371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.395218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:46.395904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.395904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.396324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.397359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:46.398477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.398477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:46.398477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:46.399123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:46.399123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:46.399333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:46.399747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:46.400058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.401043 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:46.402516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.403403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.404032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:46.404761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.405521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.405521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:46.405521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.405521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.405795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:46.406711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:46.407731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:46.408247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:46.408396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.409374 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:46.410810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.411538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.412583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:46.413208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.413639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.413639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:46.413639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.413639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.414255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:46.414780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:46.415362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:46.415700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:46.416120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:46.416476 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:46.416714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.417519 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.419294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:46.419694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.419694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:46.420619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:46.421287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:46.421871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:46.421871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:46.421871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:50.955393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:50.956237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:50.956237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:50.957144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:50.991164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:50.991645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:50.992783 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:50.993826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:50.994216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:50.994580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:50.994580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:50.994813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:50.995309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:50.995827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:50.995827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:50.995827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:51.390468 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:51.391279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:51.391379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:51.392732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:51.393135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:51.393364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.394286 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.396225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.396894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:51.397519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.397519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.398140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.398860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:51.399367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.399367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:51.399367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:51.399515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:51.399667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:51.399817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:51.400536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:51.400778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.401734 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:51.403292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.404019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.404508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:51.404885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.405328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.405328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:51.405328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.405328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.406066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:51.406629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.407380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.407380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:51.407380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.407380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:51.407380 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:51.407380 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:51.407652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:51.407702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:51.408395 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:51.408763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.409427 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:51.410803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.411356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.412303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:51.412772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.413298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.413298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:51.413298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.413298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.414024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:51.414673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.415298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.415298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:51.415298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.415298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:51.415489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:51.416184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:51.416184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:51.416519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:51.416871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:51.417270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:51.417498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.418336 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.419408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.419793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:51.420198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.420198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:51.420428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:51.420894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:51.421465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:51.421465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:51.421465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:55.956043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:55.956834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:55.957114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:55.958720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:55.996657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:55.996941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:55.997986 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:55.998884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:55.999644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:56.000611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:56.001666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.002787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:56.002787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.002787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.002787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:56.002787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.002787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:56.391711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:56.392257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:56.392330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:56.393420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:56.393719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:56.393958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.394674 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.395507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.396291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.397079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:56.398098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:56.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:56.399311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:56.399606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:56.399749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:56.399928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:54:56.400380 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:56.400608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.401228 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:56.402459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.403055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.403727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:56.404278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.404789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.404789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:56.404789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.404789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.405280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:56.405941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.406579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.406579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:56.406579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.406579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:56.407162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:56.407484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:56.407580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:56.407674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:56.408074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:56.408321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.408850 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:54:56.409473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.409938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.410244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:56.410600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:56.411063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:56.411517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.412114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:54:56.415917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:54:56.416686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:54:56.416856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:54:56.417186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:54:56.417589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:56.417834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:54:56.417992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.418661 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.420200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.420885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:54:56.421538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.421538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:54:56.421750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:54:56.422579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:54:56.423397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:54:56.423397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:54:56.423397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:00.956871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:00.957811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:00.958185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:00.959673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:00.999097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:00.999604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.000790 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.002019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.002447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:01.002785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.002785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:01.003533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.004158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.004793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:01.004793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.004793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:01.392580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:01.393407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:01.393531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:01.394898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:01.395345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:01.395616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.396543 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.397984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.398307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:01.398602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.398602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.399510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.400293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:01.401375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.401957 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:01.403490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.404546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.405088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:01.405780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.406512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:01.407347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:01.407989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:01.408352 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:01.408675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:01.409020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:01.409545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:01.409771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.410455 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:01.412185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.412784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.413723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:01.414196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.414666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.414666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:01.414666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.414666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.415755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:01.416318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.416814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.416814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:01.416814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.416814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:01.416814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:01.416960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:01.416960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:01.417208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:01.417584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:01.417956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:01.418248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.419212 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.420494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.420880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:01.421263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.421263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:01.421788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:01.422301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:01.422987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:01.422987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:01.422987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:05.957625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:05.958483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:05.958679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:05.960095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:05.998679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:05.998965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:05.999818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.000913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.001680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:06.002463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:06.003388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.004467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:06.004467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.004467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.004467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:06.004467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.004467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:06.393752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:06.394484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:06.394716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:06.396124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:06.396436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:06.396596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.397367 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.399284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.399886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:06.400500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.400500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.401209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.402314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:06.403137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.403620 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:06.404272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.405156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.405591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:06.406172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.406758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:06.407501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:06.408324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:06.409149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:06.409149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:06.409242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:06.409876 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:06.410371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.411343 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:06.412710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.413262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.414194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:06.414564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.414950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.414950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:06.414950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.414950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.416119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:06.416669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.417219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.417219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:06.417219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.417219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:06.417219 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:06.417383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:06.417497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:06.417651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:06.417880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:06.418117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:06.418276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.418715 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.419530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.419891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:06.420294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.420294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:06.420509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:06.420964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:06.421519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:06.421519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:06.421519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:10.958379 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:10.959088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:10.959472 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:10.961288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:10.997644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:10.998086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:10.999121 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.000110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.000844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:11.001585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:11.002406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.005957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:11.005957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.005957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.005957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:11.005957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.005957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:11.394628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:11.395456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:11.395586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:11.396903 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:11.397357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:11.397658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.398676 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.400331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.400764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:11.401165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.401165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.402245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.402800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:11.403394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.403394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:11.403394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:11.403394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:11.403394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:11.403394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:11.403647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:11.403906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.404442 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:11.405270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.405828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.406325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:11.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.407178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.407178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:11.407178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.407178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.407609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:11.408192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.408825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.408825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:11.408825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.408825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:11.408825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:11.409210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:11.409421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:11.409640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:11.410194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:11.410972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.411691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:11.412467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.412893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.413337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:11.413662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.414050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.414050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:11.414050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.414050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.414375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:11.414857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.415407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.415407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:11.415407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.415407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:11.417415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:11.417704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:11.417866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:11.418119 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:11.418406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:11.418629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:11.418776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.419291 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.420479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.420836 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:11.421204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.421204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:11.421758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:11.422246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:11.422781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:11.422781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:11.422781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:15.959149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:15.960168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:15.960168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:15.961795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:16.002247 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.002697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.003723 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.004732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.005499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:16.006322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:16.007248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.008240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:16.008240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.008240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.008240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:16.008240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.008240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:16.395486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:16.396302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:16.396458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:16.398036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.398543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.398820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.399863 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.401613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.402303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:16.402823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.402823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.403272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.404093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:16.404946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.404946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:16.404946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:16.405509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:16.405623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:16.405725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:16.406279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.406594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.407354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:16.408185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.408716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.409221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:16.409606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.410019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.410019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:16.410019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.410019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:16.410019 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.411225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:16.411827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:16.412402 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:16.412738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.412976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.413668 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:16.415138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.415569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.416658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:16.417092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.417490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.417490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:16.417490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.417490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.418163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:16.418733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:16.419260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:16.419759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.420174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:16.420419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.421221 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.422140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.422523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:16.422890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.422890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:16.423156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:16.423624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:16.424133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:16.424133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:16.424133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:20.959935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:20.961313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:20.961632 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:20.963505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:21.000382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.000889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.001772 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.003024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.003582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:21.003998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.003998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:21.004138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.004697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.005454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:21.005454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.005454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:21.397517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:21.398158 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:21.398321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:21.399716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.400204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.400450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.401438 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.403307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.404342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:21.404992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.404992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.405267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.405797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:21.406451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.406451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:21.406451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:21.406451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:21.406451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:21.406451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:21.407309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.407309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.408207 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:21.409036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.410420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:21.411597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.412711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:21.413369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.414387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:21.415258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:21.415258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:21.415258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:21.415258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:21.415258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.415258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.415625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:21.416497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.417070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.417525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:21.417893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:21.418255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.418780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:21.419091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:21.419486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:21.419686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:21.420145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:21.420854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.421330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:21.421696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.422711 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.423840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.424539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:21.425234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:21.425942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:21.427489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:21.427489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:21.427489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:21.427489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:21.427489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:21.427489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:25.960759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:25.961863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:25.962136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:25.963805 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:26.002134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.002467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.003415 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.004420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.005366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:26.006214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:26.007372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.008543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:26.008543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.008543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.008543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:26.008543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.008543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:26.398342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:26.398881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:26.398881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:26.400405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.400838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.401132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.402049 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.403686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.404471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.405320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:26.407050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:26.407766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.408112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.408346 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:26.409382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.410200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.410653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:26.411200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.411828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.411828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:26.411828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.411828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.412073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:26.412744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:26.413606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:26.413975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.413975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.414486 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:26.415856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.416234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.417472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.417772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.418071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:26.418071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.418071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.419383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:26.419747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.420124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.420124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:26.420124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.420124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:26.420124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:26.420242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:26.420440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:26.420720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:26.421692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.421965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:26.422098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.422606 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.424134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.424505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:26.424865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.424865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:26.425714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:26.426196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:26.426824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:26.426824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:26.426824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:30.961769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:30.962462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:30.962462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:30.963755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:31.005286 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.006172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.006915 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.008480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.008910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:31.009360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.009360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:31.009547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.010041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.010564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:31.010564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.010564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:31.399382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:31.399735 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:31.399864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:31.401316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.401737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.401984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.402957 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.404638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.405230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:31.405864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.405864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.406269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.407051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:31.407934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.408384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.408931 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:31.410231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.410937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.411705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:31.412058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.412452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.412452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:31.412452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.412452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.412993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:31.413512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:31.414095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:31.414356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.414682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.415371 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:31.416162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.416630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.417011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.417379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.417791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:31.417791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.417791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.418110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:31.418599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.419135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.419135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:31.419135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.419135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:31.420245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:31.420727 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:31.420970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:31.421293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:31.421790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.422218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:31.422464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.423320 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.425187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.425567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:31.425933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.425933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:31.426862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:31.427517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:31.428063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:31.428063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:31.428063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:35.962707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:35.963740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:35.963740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:35.964410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:35.966382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:55:36.010447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Traceback (most recent call last):
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.011584 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.012684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.013055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:36.013390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.013390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-36435079-9283-4fbf-b78b-05a805a67deb None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:36.013638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.014299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.014775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:36.014775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.014775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:36.400242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:36.400594 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:36.400769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:36.402196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.402619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.402884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.404030 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.405626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.406208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:36.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.406745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.407465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.408734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:36.409521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.409911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.410340 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:36.411755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.412196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.413091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:36.413426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.413785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.413785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:36.413785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.413785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.415014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:36.415461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:36.415987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:36.416673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.417107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.417834 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:36.418442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.418851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.419207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.419521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.419937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:36.419937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.419937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.421021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:36.421461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.422051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.422051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:36.422051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.422051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:36.422051 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:36.422773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:36.422773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:36.422773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:36.423236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.423630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:36.423630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.424201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.425124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.425722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:36.426407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:36.427165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:36.428039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:36.428039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:36.428039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:36.428039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:36.428039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:36.428039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:40.963401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:40.964526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:40.964851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:40.966308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:41.007920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.008429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.009155 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.010458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.011170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:41.011980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:41.012841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:41.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:41.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.013812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:41.414940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:41.414940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:41.415319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:41.415808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:41.416127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:41.416629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:41.417212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:41.417600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:41.417815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.418361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:41.418559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.418860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:41.419088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.419933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:41.420205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.420536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:41.420824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.422459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.422794 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.423180 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:41.424357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.424790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.425439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:41.425439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.425439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:41.425439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:41.425439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:41.426461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.427055 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.427981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:41.428381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.428790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.428790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:41.428790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.428790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:41.428790 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:41.429884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.430381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.431341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:41.432113 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.432818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.432818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:41.432818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.432818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:41.432818 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.433636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:41.434151 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.434707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.434707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:41.434707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.434707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:41.434707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.435649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:41.436629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.437329 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.438072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.438429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:41.438767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.438767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:41.439111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:41.439618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:41.440181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:41.440181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:41.440181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:45.964261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:45.965294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:45.965294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:45.966359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:46.007079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.007596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.008635 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.009936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.010608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:46.011428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:46.012335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.013277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:46.013277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.013277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.013277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:46.013277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.013277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:46.415479 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:46.415801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:46.415941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:46.416127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:46.417027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:46.418709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.418709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:46.418709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:46.418709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.419120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.419813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.420618 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:46.421298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.422675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.423472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:46.424197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:46.424717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:46.425250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:46.425514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:46.425969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.426634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:46.426784 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:46.427030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.427918 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:46.428547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:46.429851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:46.430416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.431820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:46.432484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.432960 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.433590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:46.434148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.434712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.434712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:46.434712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.434712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:46.434712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:46.435923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:46.436411 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.438185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:46.438539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.438897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.438897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:46.438897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.438897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:46.438897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:46.441516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.442040 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task update_mac_aging_settings failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.442948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.443378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:46.443740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.443740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:46.444607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:46.445143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:46.445674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:46.445674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:46.445674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:50.964895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:50.965571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:50.965571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:50.966081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:50.998276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:50.998543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:50.999231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.000194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.000527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:51.000838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.000838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:51.000955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.001460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.001900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:51.001900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.001900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:51.416333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:51.417641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:51.417641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:51.418483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:51.418952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:51.419602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.420047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:51.420664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:51.421345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:51.421580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.424946 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:51.425185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.429717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:51.430303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.430303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:51.430303 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:51.430421 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:51.430690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:51.431715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.432078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.432435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:51.432435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.432435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:51.432435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.432674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:51.433114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.433447 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:51.433974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:51.434497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:51.436154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.437571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:51.438702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:51.439131 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.439498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.439946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:51.440631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.441228 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:51.441485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.443563 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.444317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:51.445222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:51.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.445902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:51.445902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.445902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.445902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:51.445902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.445902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:51.446174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.446752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.447491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:51.447491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.447491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:51.476858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:51.478246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:51.478979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:51.478979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:51.478979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:51.478979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:55.965506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:55.966376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:55.966515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:55.967756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:56.009541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.009795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.010642 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.011690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.011969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:56.012235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.012235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:56.013177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.013516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.013880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:56.013880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.013880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:56.417483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:56.417811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:56.417957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:56.419397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.421047 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.422779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.423338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:56.423811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.423811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:56.425489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.427088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.427629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:56.428184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.428184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:56.428184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:56.428184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:56.428184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:55:56.428184 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:56.428391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:56.428391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:56.429231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.429534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.430427 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:56.431846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.432313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.433046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:56.433431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.433821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.433821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:56.433821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.433821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:56.433821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.433821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.435315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:56.435815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:55:56.436360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:55:56.436599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:55:56.436949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.437246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:55:56.437421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.437957 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.438948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.439409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.439766 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:56.442033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.442666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:55:56.443240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.443672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.444119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.444472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:55:56.444805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:55:56.445314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:55:56.445827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:55:56.445827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:55:56.445827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:55:56.445827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:00.966242 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:00.966897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:00.966897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:00.967645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:01.002720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.003081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.004152 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.005336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.005774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:01.006187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.006187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:01.007060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.008124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.009174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:01.009174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.009174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:01.418195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:01.420442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:01.420530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:01.421669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.421942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.422047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.422383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:01.423563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:01.423563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:01.423563 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.424515 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.426169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.426580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:01.426940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.426940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:01.426940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.428486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.429025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:01.429589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.429589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:01.429589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.429589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:01.429589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:01.429589 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:01.429795 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.430765 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:01.431755 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.432298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:01.433094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.434343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:01.434860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.435676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.435676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:01.435676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.435676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:01.435676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.436181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:01.438303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.438870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.439466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.439949 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:01.440428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:01.441353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:01.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.442512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.443060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.443538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:01.443538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.443538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:01.443538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.443772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:01.444603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:01.445381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.445684 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.446480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.446894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:01.447299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:01.447743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:01.448311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:01.448311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:01.448311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:01.448311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:01.448311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:01.448311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:05.967132 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:05.967920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:05.968280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:05.969667 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:06.008109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.008356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Traceback (most recent call last):
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.009455 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.010760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.011502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-027f4377-1826-42c8-8b64-80a1ce00dbba None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:06.011994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:06.012559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.013729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:06.013729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.013729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.013729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:06.013729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.013729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:06.420778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:06.421498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:06.421640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:06.423062 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.423437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.423727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.424846 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:06.425550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.427312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.428286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:06.429163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:06.430154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:06.431497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.432495 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:06.433027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:06.433585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.433986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:06.434484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.434956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.434956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:06.434956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.434956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:06.434956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:06.435638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.437203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.437602 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:06.438653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:06.439271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.440270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.440731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.441102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:06.441102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.441102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:06.441102 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.441880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:06.442361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.442837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:06.443075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.443685 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.445509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.445843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:06.446140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.446140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:06.447533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:06.448033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:06.448513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:06.448513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:06.448513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:10.967733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:10.968631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:10.968755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:10.970054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:11.001149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.001988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.003708 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.004172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.004787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:11.005389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:11.006213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.007548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:11.007548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.007548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.007548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:11.007548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.007548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:11.421526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:11.423145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:11.423145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:11.424097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.424568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.424812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.426468 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.428165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.428804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:11.429417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.429417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:11.430025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.431912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.432404 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:11.432890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.432890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:11.432890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:11.432890 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:11.433227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:11.433483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:11.434215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.434499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.435170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:11.435243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:11.435377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.435743 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:11.437882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:11.438324 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.439561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:11.439913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.440288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.440288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:11.440288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.440288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:11.440288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.440288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:11.443092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.443554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:11.444021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.444262 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:11.444467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:11.445440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:11.445874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.447107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.447461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.447808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:11.447808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.447808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:11.447808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.447808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:11.447808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.449290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:11.449750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.450239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.450239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:11.450239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.450239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Traceback (most recent call last):
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.450574 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.453388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:11.453718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.453718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-19cffbb1-135b-4d5a-bb9b-af928791f312 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:11.455444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:11.455919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:11.456439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:11.456439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:11.456439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:15.969432 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:15.970643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:15.970643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:15.972195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:16.006963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.007433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.008789 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.009979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.010442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:16.010839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.010839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:16.011018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.011554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.012166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:16.012166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.012166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:16.422487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:16.423177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:16.423265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:16.425134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.425373 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.425607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.426603 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.427965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.428538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:16.429210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.429210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.429479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.430045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:16.430566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.430566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:16.430566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:16.431277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:16.431473 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:16.431671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:16.432285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.432622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.433648 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:16.435016 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.435546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.436626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:16.437040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.437464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.437464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:16.437464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.437464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:16.437464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:16.438373 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.438822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:16.439363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:16.439681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.439681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.440126 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:16.440898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.441325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.441944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:16.442330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.442750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.442750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:16.442750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.442750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:16.442750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.443795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:16.444316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.444838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.444838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:16.444838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.444838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:16.444838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:16.444838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:16.445108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:16.445521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.445897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:16.446202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.447077 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.448951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.449388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:16.449772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.449772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:16.450197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:16.450659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:16.451182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:16.451182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:16.451182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:20.970175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:20.970940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:20.971113 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:20.972695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:21.010965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.011447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.012456 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.013359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.013979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:21.014545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:21.015219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.016095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:21.016095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.016095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.016095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:21.016095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.016095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:21.423322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:21.423825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:21.423938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:21.425369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.425804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.426164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.427134 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.428845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.429546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:21.430172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.430172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.430899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.431778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:21.432713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.432713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:21.432713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:21.432713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:21.432713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:21.432713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:21.433147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.433264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.433962 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:21.435499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.436263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.436940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:21.437605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.437985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.437985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:21.437985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.437985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:21.438740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.439263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:21.439802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:21.440078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:21.440078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:21.440969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.441473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.442263 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:21.443249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.444137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.444619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:21.445435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:21.446273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:21.447324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:21.448331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:21.448959 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.449437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.450213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:21.450912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:21.451844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:21.452634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:21.452634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:21.452634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:21.452634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:21.452634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:21.452634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:25.970997 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:25.972082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:25.972559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:25.973853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:26.017559 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.017847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.018818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.020029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.020831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:26.021509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:26.022344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.023847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:26.023847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.023847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.023847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:26.023847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.023847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:26.424290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:26.424870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:26.424967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:26.426291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.426676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.427011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.428712 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.429825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.430244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:26.430567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.430567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.430902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.431638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:26.432438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.432438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:26.432438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:26.432438 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:26.432674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:26.432853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:26.433497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.433783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.434468 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:26.436484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.437326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.437799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:26.438439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:26.439105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.439929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:26.441014 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:26.441746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:26.441746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:26.441746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.441746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.442227 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:26.443680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.444263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.445427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:26.445862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.446279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.446279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:26.446279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.446279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.447494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:26.448152 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:26.448699 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:26.449195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.449368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:26.449488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.449818 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.450530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:26.450996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:26.451346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:26.451781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:26.452261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:26.452261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:26.452261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:30.971751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:30.973059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:30.973059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:30.973428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:31.000931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.001616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.002782 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.004364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.005321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:31.005740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.005740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:31.006183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.006591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.006950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:31.006950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.006950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:31.425058 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:31.425833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:31.426136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:31.427510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.427848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.428076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.428936 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.430786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.431247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:31.431611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.431611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.432264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.432760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:31.433333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.433333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:31.433333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:31.433333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:31.433464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:31.433694 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:31.434323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.434640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.435540 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:31.437136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.437647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.438575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:31.438987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.439448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.439448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:31.439448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.439448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:31.440338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.440796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:31.441440 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:31.441716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:31.441716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:31.442238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.442522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.443322 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:31.444205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.444616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.445358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:31.445721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.446193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.446193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:31.446193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.446193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.446544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:31.447231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.447775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.447775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:31.447775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.447775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:31.448650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:31.448934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:31.449038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:31.449109 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:31.449293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.449441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:31.449555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.449855 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.450417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:31.450829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:31.451383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:31.452135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:31.452789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:31.452789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:31.452789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:35.972762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:35.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:35.973751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:35.974662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:35.976921 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:56:36.016962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.016962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.017929 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.018750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.019707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:36.020400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:36.021264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.022296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:36.022296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.022296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.022296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:36.022296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.022296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:36.426353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:36.427295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:36.427533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:36.428676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.429097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.429366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.430329 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.431557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.432210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:36.432759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:36.433509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:36.434394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:36.434959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.434959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.435686 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:36.437074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.437662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.438338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:36.438730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.439172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.439172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:36.439172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.439172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.439928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:36.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.441073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.441073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:36.441073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.441073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:36.441322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:36.441854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:36.442162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:36.442418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:36.442938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.443987 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:36.444687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.445188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.445648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:36.445988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.446398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.446398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:36.446398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.446398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.446822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:36.447366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.447897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.447897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:36.447897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.447897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:36.449220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:36.449628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:36.449787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:36.450024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:36.450493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.451011 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:36.451332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.452422 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.453586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:36.454061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.454396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:36.454591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:36.455175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:36.455699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:36.455699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:40.973374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:40.974072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:40.974072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:40.975407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:41.015112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.015444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.016399 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.017523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.018413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:41.019184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:41.020123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.021323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:41.021323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.021323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.021323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:41.021323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.021323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:41.427503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:41.428333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:41.428519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:41.429698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.430160 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.430418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.431493 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.433162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.433770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:41.434339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.434339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:41.435282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.436129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:41.436952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:41.437340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.437536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.438312 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:41.439123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.439547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.439898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:41.440273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.440629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.440629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:41.440629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.440629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:41.440885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.441330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:41.442415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:41.442836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:41.443169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:41.443500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:41.444739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.445201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.446156 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:41.448153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.448638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.449870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:41.450772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.451566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.451566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:41.451566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.451566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.451872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:41.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:41.453657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:41.454192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.454192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:41.454192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.454788 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.455654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.456060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:41.456430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.456430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:41.456717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:41.457273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:41.457828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:41.457828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:41.457828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:45.496330 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:45.497349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:45.497349 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:45.507026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 02:56:45.507179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 02:56:45.507284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:56:45.542926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.543541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.544214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.544439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.545837 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.548076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.549598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:45.549598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.549598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:45.549598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Fixing resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:56:45.579611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, resource={'id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['97900556-2eeb-4240-89bd-ed8258b9022d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:48Z', 'updated_at': '2026-04-13T02:30:49Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9202}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.579847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.580519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8120b9cd-5d7c-4d4d-9a57-22a6c716f312, external_ids_update=None, columns={'tag': 9202}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.580690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.581659 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Failed to fix resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.582975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.583758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.584485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:45.584485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.584485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:45.584485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:56:45.611916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.612356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.612968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.613234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.614135 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.615402 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.616043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.616830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:45.616830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.616830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:45.616830 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:56:45.648336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.648746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.649557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.649983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.651127 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.652698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.653317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.653941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:45.653941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.653941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:45.653941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Fixing resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:56:45.733464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, resource={'id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'enable_snat': True}, 'gw_port_id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-exgz1', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'ip_address': '172.24.5.236', 'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, {'ip_address': '2001:db8::2c9', 'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 3, 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46'}, resource_type=routers, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.733787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.733787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.733941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.734035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.734606 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:45.734870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Unable to update router 530b037d-a47e-4f63-bd55-041139eeee2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Failed to fix resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 13 02:56:45.736057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:45.736570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:45.737025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Fixing resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 02:56:45.853140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, resource={'id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'name': '', 'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'project_id': '', 'mac_address': 'fa:16:3e:92:a4:24', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000165409', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.853686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.236/24', '2001:db8::2c9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.854289 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3ec69888-b9c9-4830-9d48-853dd81cf68b, lrouter_port=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:45.854700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.855977 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Maintenance task: Failed to fix resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.857416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:45.857904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:45.858514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:45.858514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:45.858514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:45.858514 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] OVN maintenance task check_for_inconsistencies finished in 0.361 seconds
Apr 13 02:56:45.974666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:45.975230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:45.975486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:45.976879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:46.019093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.019812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.020661 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.021815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.022275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:46.022655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.022655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:46.022899 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.023748 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.024955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:46.024955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.024955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:46.428363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:46.429091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:46.429335 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:46.430756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.431201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.431490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.432601 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.434412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.435145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:46.435844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.435844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:46.436581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.437179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.437753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:46.437753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.437753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:46.437753 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:46.437753 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:46.437753 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:46.437958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:46.438406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.438630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.439696 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:46.441447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.442047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.442847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:46.443336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.443731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.443731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:46.443731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.443731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.446224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:46.446737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:46.447460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:46.448933 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.449235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.449913 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:46.451188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.451744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.452542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.452968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.453387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:46.453387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.453387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:46.453387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.454445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:46.454944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:46.455617 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:46.455848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.455848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:46.456121 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.456900 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.458026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.458416 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:46.458759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.458759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:46.459603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:46.460792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:46.461789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:46.461789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:46.461789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:50.975529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:50.976025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:50.976274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:50.977641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:51.013301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.013774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.014804 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.015869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.016226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:51.016517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.016517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:51.016958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.017366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.017807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:51.017807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.017807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:51.429218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:51.430146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:51.430356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:51.431814 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.432256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.432522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.433646 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.435389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.436158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:51.436797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.436797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:51.438018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.438530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:51.439120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:51.439461 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.439874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.440512 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:51.441341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.441889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.442262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:51.442661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.443115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.443115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:51.443115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.443115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:51.443257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.443768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:51.446860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:51.447651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:51.447825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:51.448106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:51.448604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.448859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.449693 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:51.452252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.452770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.454239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:51.454648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.455140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.455140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:51.455140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.455140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:51.455140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.457295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:51.457903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:51.458505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:51.458797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.459015 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:51.459237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.459804 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:51.462167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.462592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:51.463430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:51.463964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:51.464575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:51.464575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:55.976351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:55.977245 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:55.977367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:55.978714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:56.014112 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.014429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.015600 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.016329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.017029 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:56.017751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:56.018636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.019641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:56.019641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.019641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.019641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:56.019641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.019641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:56.430097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:56.430538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:56.430688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:56.432124 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.432527 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.432785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.433863 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.435524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.436169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:56.436715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.436715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:56.437478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.438667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:56:56.439604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.440290 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:56.440932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:56.442086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.442700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.443270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:56.443824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.444557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.445160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:56.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:56.448032 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:56.448280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:56.448525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:56.449035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.449320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Traceback (most recent call last):
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.449971 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:56:56.451597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.452444 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.453494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:56.453856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.454364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.454364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:56.454364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.454364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.455637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:56.456239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.456759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.456759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:56.456759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.456759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:56:56.456759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:56:56.456759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:56:56.457024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:56:56.457094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:56:56.457533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.457791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:56:56.457919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Traceback (most recent call last):
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.458591 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.459843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.460729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c60c4829-22a4-49f9-91de-5f8a86652208 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:56:56.461485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:56:56.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:56:56.463321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:56:56.463321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:56:56.463321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:56:56.463321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:56:56.463321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:56:56.463321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:00.977390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:00.978420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:00.978420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:00.979696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:01.022768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.023240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.024262 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.025221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.025858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:01.026468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:01.027189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.027976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:01.027976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.027976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.027976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:01.027976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.027976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:01.431486 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:01.432236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:01.432433 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:01.433866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.434255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.434532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.435530 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.437316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.438059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:01.438635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.438635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:01.439278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.440244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:01.441284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:01.441697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.441697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.442253 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:01.443316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.443805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.444331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:01.444697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.445141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.445141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:01.445141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.445141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:01.445734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.446331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.446807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:01.447957 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:01.448584 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:01.448780 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:01.449078 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:01.449600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.449975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.450708 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:01.451971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.452815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.453565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:01.454204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.454906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.454906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:01.454906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.454906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.455493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:01.456143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.456642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.456642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:01.456642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.456642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:01.456642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:01.457233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:01.457435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:01.457736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:01.458313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.458678 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:01.458941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Traceback (most recent call last):
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.459785 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.461534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.462173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:01.462793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.462793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-48020af6-06ba-4f04-93da-a1638e3d6ab0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:01.463339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:01.464116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:01.465054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:01.465054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:01.465054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:05.978197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:05.979153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:05.979258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:05.980672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:06.017658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.018217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.019241 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.020320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.020900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:06.021514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:06.022212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.022977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:06.022977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.022977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.022977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:06.022977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.022977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:06.432279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:06.433037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:06.433287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:06.434653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.435040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.435312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.436390 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.438106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.438911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:06.439640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.439640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:06.439783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.440807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:06.441895 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.442588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.443928 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:06.445507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.446059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.446889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:06.447342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.447773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.447773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:06.447773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.447773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:06.448391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.448970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:06.449561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:06.449848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:06.450155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:06.450684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.450897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.451735 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:06.453319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.453727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.454739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:06.455109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.455507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.455507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:06.455507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.455507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.456657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:06.457224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.457738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.457738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:06.457738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.457738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:06.457738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:06.458017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:06.458317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:06.458625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:06.459173 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.459529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:06.459789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.460591 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.461802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.462228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:06.462596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.462596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:06.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:06.463698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:06.464235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:06.464235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:06.464235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:10.979798 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:10.981114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:10.981305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:10.982660 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:11.020210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.020674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.021732 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.022860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.023682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:11.024498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:11.025452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.026697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:11.026697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.026697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.026697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:11.026697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.026697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:11.433059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:11.433802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:11.434039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:11.435236 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.435642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.436051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.437050 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.438833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.439509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:11.440117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:11.440992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:11.441835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:11.442573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:11.442774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:11.442976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:11.443390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.443615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.444299 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:11.445112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.445550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.445907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:11.446324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.446730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.446730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:11.446730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.446730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:11.446971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.447429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.447892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:11.451381 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:11.452215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:11.452416 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:11.452672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:11.453265 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.453609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.454588 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:11.456027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.456845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.457552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:11.458319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.459065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.459065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:11.459065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.459065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.459825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:11.460896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:11.461732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:11.462230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.462230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:11.462643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.463195 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.464328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:11.464710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.465066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:11.465996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:11.466559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:11.467133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:11.467133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:15.980657 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:15.981606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:15.981778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:15.983278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:16.020385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.020915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.022195 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.023262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.024189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:16.025060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:16.026057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.027243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:16.027243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.027243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.027243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:16.027243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.027243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:16.433884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:16.434698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:16.434900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:16.436302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.436776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.437141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.438305 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.440192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.441550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:16.442593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:16.443808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:16.445685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.446332 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:16.446896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:16.447419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.447948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.448408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:16.448834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.456084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:16.456978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:16.457558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:16.458147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:16.458767 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:16.459408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:16.459905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.460298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.461042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:16.461494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:16.461494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:16.461637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:16.461991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:16.462969 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.463484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:16.463973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.464846 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.465766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.466163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:16.466507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.466507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:16.466935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:16.467383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:16.467867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:16.467867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:16.467867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:20.981477 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:20.982287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:20.982415 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:20.983822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:21.020951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.021426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.022346 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.023169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.023863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:21.024603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:21.025576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.026469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:21.026469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.026469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.026469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:21.026469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.026469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:21.434700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:21.435537 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:21.435741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:21.436765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.437072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.437314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.438498 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.440240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.440878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:21.441498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:21.442395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:21.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:21.444133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:21.444488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:21.444488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:21.445080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.445284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.445959 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:21.447649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:21.448885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.449624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.450249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:21.450804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.451558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.452413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:21.452725 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:21.453297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:21.453488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:21.453712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:21.454354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.454695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.455564 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:21.457041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.457929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.458579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:21.459132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.459791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.459791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:21.459791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.459791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.460519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:21.461306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.461831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.461831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:21.461831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.461831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:21.461831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:21.461831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:21.462065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:21.462328 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:21.462794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.463201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:21.463436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.464233 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.465726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.466102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:21.466453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.466453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:21.467430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:21.467904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:21.468440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:21.468440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:21.468440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:25.982224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:25.983045 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:25.983251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:25.984676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:26.026846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.026988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.027536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.028280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.028968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:26.029706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:26.030575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.031561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:26.031561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.031561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.031561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:26.031561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.031561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:26.435544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:26.436524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:26.436752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:26.438075 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.438393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.438607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.439450 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.441081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.441643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:26.442234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.442234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:26.442764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.443621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:26.444825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:26.445316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.445571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.446316 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:26.447789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.448654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.449164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:26.449815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.450520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.450520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:26.450520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.450520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:26.450862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.451371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.451950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:26.452931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:26.453764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:26.454016 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:26.454436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:26.454952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.455192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.455901 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:26.457592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.458108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.459082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:26.459623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.459984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.459984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:26.459984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.459984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.460301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:26.460837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.461421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.461421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:26.461421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.461421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:26.461788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:26.462375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:26.462505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:26.462756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:26.463682 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.464057 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:26.464239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.464940 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.466088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.466431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:26.466765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.466765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:26.467178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:26.467609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:26.468104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:26.468104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:26.468104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:30.982995 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:30.983721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:30.983926 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:30.985448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:31.019934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.020431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.021460 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.022637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.023344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:31.024055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:31.024852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.025812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:31.025812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.025812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.025812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:31.025812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.025812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:31.436466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:31.437308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:31.437569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:31.439343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.439960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.440263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.441349 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.442793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.443432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:31.443893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:31.444368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:31.444887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:31.445151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:31.445190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:31.445801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.446198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.447073 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:31.448519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.449037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.449923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:31.450370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.450784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.450784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:31.450784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.450784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:31.451727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.452221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.452765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:31.453342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:31.454159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:31.454509 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:31.454722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:31.455225 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.455472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.456333 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:31.457856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.458341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.459128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:31.459477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.459887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.459887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:31.459887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.459887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.460265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:31.460819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.461375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.461375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:31.461375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.461375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:31.462298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:31.462968 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:31.463471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:31.463834 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:31.464475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.464962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:31.465259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.466153 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.467718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.468132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:31.468499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.468499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:31.469442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:31.469942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:31.470519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:31.470519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:31.470519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:35.984038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:35.985083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:35.985083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:35.985624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:35.988036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=434 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:57:36.034745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.035531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.036413 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.037541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.038171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:36.038843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:36.039729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.040649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:36.040649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.040649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.040649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:36.040649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.040649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:36.437292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:36.437985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:36.438256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:36.440089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.440512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.440766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.441851 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.443739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.444354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:36.444931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.444931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:36.445554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.446308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.447064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:36.447064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.447064 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:36.447064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:36.447064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:36.447319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:36.447376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:36.447943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.448208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.448842 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:36.450325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.451114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.452855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:36.453293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.453714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.453714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:36.453714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.453714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:36.455610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.456134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:36.456677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:36.457029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:36.457350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.457601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.458312 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:36.459269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.459821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.460198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:36.460524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.460904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.460904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:36.460904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.460904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.461855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:36.462528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.463144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.463144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:36.463144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.463144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:36.463144 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:36.464181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:36.464307 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:36.464545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:36.465268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.465613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:36.465873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.466669 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.467842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.468258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:36.468656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.468656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:36.468896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:36.469425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:36.470100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:36.470100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:36.470100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:40.988782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:40.989902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:40.989902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:40.993991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:41.038974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.039172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.039639 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.040321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.040752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:41.041226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:41.041718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.042396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:41.042396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.042396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.042396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:41.042396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.042396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:41.438080 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:41.438707 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:41.438845 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:41.441107 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.441423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.441601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.442326 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.443604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.443996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:41.444403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.444403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:41.445323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.445798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.446358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:41.446358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.446358 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:41.446644 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:41.447305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:41.447523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:41.447733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:41.448164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.448370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.448917 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:41.449891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.450721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.451105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:41.451454 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.451825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.451825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:41.451825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.451825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:41.452263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.452758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.453347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:41.456382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:41.457079 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:41.457281 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:41.457510 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:41.458089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.458436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.459274 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:41.460687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.461352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.462471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:41.463506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:41.464196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:41.464970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:41.465941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:41.467816 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.468477 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.468477 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:41.468477 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.468477 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.468882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.469621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:41.470139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:41.470550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:41.471058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:41.471058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:41.471058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:41.471058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:41.471058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:41.471058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:45.990665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:45.991196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:45.991446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:45.994942 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:46.031747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.032217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.033187 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.035150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.035808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:46.036465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.036465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:46.036949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.037808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.038762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:46.038762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.038762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:46.438854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:46.439347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:46.439538 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:46.440874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.441287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.441549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.442613 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.444248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.445051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:46.445681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.445681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:46.445791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.446556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.447360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:46.447360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.447360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:46.447454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:46.447849 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:46.448083 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:46.448283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:46.448849 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.449200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.449983 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:46.451068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.451560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.452238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:46.452712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.453118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.453118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:46.453118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.453118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:46.453290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.454109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.454647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:46.456731 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:46.457276 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:46.457502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:46.457801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:46.458428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.458818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.459691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:46.461107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.461556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.462813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:46.463205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.463605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.463605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:46.463605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.463605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.464084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:46.464620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.465140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.465140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:46.465140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.465140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:46.465246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:46.466081 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:46.466308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:46.466631 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:46.467125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.467348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:46.467493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Traceback (most recent call last):
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.468306 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.470420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.470815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:46.471193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.471193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ad0b7811-a9d5-4ffa-8be3-fae998530fb0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:46.472425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:46.472984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:46.473527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:46.473527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:46.473527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:50.991700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:50.992637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:50.992637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:50.993924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:51.030619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.031061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.032055 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.032849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.033461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:51.033969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:51.034505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.035129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:51.035129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.035129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.035129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:51.035129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.035129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:51.439669 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:51.440333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:51.440474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:51.441513 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.441872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.442169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.443078 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.443869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.444692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:51.445407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:51.446225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:51.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:51.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.447178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:51.447863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:51.448162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:51.448257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:51.448455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:51.449195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.449577 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.450526 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:51.451611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.452183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.452565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:51.452993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.453398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.453398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:51.453398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.453398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:51.453658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.454305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.455074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:51.457147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:51.457596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:51.457791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:51.458030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:51.458493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.458786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.459568 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:51.460838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.461413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.462306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:51.462733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.463191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.463191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:51.463191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.463191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.464019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:51.464630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.465256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.465256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:51.465256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.465256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:51.465692 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:51.466368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:51.467099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:51.467099 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:51.467426 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.467654 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:51.467815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.468549 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.469572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.470184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:51.470779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:51.471541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:51.472445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:51.472445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:51.472445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:51.472445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:51.472445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:51.472445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:55.992362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:55.993434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:55.993434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:55.994680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:56.032511 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.033164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.034487 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.035584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.036050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:56.036482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:56.037021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:56.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:56.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.037906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:56.440448 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:56.440912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:56.441096 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:56.442524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.442928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.443217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.444253 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.445886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.446551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:56.447131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.447131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:56.447607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.448371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:56.449306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:57:56.449714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.449795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.450536 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:56.451431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:56.452302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.453076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.453750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:56.454407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.454885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.455647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:56.457458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:56.457685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:56.457794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:56.458207 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:56.458439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.458792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.459546 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:57:56.460323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.460770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.461245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:56.461623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.462027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.462027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:56.462027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.462027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.462293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:56.462774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.463331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.463331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:56.463331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.463331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:57:56.466098 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:57:56.466859 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:57:56.467123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:57:56.467475 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:57:56.468104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.468457 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:57:56.468690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.469974 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.471171 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.471540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:57:56.471908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.471908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:57:56.472123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:57:56.472582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:57:56.473345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:57:56.473345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:57:56.473345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:00.993371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:00.994608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:00.994608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:00.995458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:01.032404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.033063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.034139 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.035118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.035544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:01.035992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:01.036534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.037288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:01.037288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.037288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.037288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:01.037288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.037288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:01.441162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:01.441832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:01.441961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:01.443458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.443726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.443882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.444892 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.446956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.447732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:01.448403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:01.449258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:01.450604 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:01.451270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.451270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.451607 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:01.452419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.452909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.453230 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:01.453574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.453933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.453933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:01.453933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.453933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:01.454269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.455075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.455599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:01.457948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:01.458406 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:01.458495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:01.458717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:01.459387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.459854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.461135 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:01.461894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.462496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.462834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:01.463306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.463680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.463680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:01.463680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.463680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.463931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:01.464775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.465310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.465310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:01.465310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.465310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:01.466970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:01.467524 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:01.467733 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:01.468142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:01.468754 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.469446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:01.469549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.470697 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.472640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.473093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:01.473445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.473445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:01.474412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:01.475053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:01.475568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:01.475568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:01.475568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:05.994364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:05.995240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:05.995430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:05.996819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:06.038878 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.039517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.040967 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.042540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.043142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:06.043717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.043717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:06.043827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.044583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.045496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:06.045496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.045496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:06.442067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:06.442917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:06.443149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:06.444574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.444963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.445321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.446352 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.447965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.448679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:06.449514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.449514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:06.449728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.450634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:06.451608 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.452292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.452833 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:06.453682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.454228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.454560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:06.454966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.455425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.455425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:06.455425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.455425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:06.455619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.456093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.456653 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:06.458495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:06.458806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:06.458920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:06.459085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:06.459530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.459883 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.460639 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:06.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.461946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.462383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:06.462763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.463220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.463220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:06.463220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.463220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.463527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:06.464099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.464632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.464632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:06.464632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.464632 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:06.467687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:06.468418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:06.468616 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:06.468923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:06.469539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.469901 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:06.470186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.471080 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.472662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.473060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:06.473475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.473475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:06.474895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:06.475441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:06.475975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:06.475975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:06.475975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:10.995500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:11.007590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:11.007590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:11.008931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:11.047491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.047955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.048985 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.049907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.050511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:11.051114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:11.051830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.052731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:11.052731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.052731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.052731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:11.052731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.052731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:11.443095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:11.443843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:11.443917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:11.445059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.445342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.445483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Traceback (most recent call last):
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.446200 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.447542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.447921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:11.449190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.449190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ecdeea65-f853-423d-a07a-93b1a997fe7a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:11.450093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.450990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.452243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:11.452243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.452243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:11.452243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:11.452701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:11.452799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:11.452967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:11.453613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.453985 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.457043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:11.457611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.458122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.458499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:11.458875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:11.459298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:11.459852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.461726 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:11.462253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.462660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.462914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:11.463280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.463682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.463682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:11.463682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.463682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.464507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:11.465040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.465385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.465385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:11.465385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.465385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:11.468068 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:11.468743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:11.468939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:11.469257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:11.469676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.469902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:11.470082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.470718 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.472448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.472861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:11.473274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.473274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:11.474021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:11.474487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:11.475027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:11.475027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:11.475027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:16.006279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:16.007355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:16.007355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:16.008750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:16.043146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.043575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.044624 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.045495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.046213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:16.046872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:16.047621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.048525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:16.048525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.048525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.048525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:16.048525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.048525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:16.444481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:16.445213 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:16.445348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:16.446732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.447166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.447411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.448368 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.450978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.451804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:16.452908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:16.454194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:16.454792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:16.454990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.455320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.456185 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:16.457877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.458508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.459037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:16.459515 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.460065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.460065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:16.460065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.460065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:16.460223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.460803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:16.461443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:16.461877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:16.461877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.461877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.462201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:16.462990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.463559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.463884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:16.464368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.464869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.464869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:16.464869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.464869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.465076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:16.465697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.466399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.466399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:16.466399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.466399 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:16.468497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:16.469170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:16.469368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:16.469680 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:16.470125 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.470366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:16.470513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.471167 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.472741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.473178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:16.473545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.473545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:16.474429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:16.474882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:16.475442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:16.475442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:16.475442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:21.006918 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:21.007853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:21.007853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:21.008959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:21.046894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.047362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.048421 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.049686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.050628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:21.051396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:21.052244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.053332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:21.053332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.053332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.053332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:21.053332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.053332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:21.445596 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:21.445954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:21.446290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:21.448456 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.449111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.449863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.451420 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.453843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.454518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:21.455231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.455231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:21.456602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.457591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:21.458561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.459267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.459570 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:21.460644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.461394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.462081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:21.462674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:21.464715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.466079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:21.468460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.469864 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:21.470468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.471255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.471825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:21.472484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:21.473237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:21.484246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:21.485233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Traceback (most recent call last):
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.485611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:21.486129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-363db1ed-4158-4c7a-a5bd-327d2c188281 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:21.486575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:21.486925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:21.487517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:26.007752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:26.008693 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:26.008853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:26.010240 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:26.047394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.047950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.048925 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.049828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.050445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:26.051056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:26.051779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.052359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:26.052359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.052359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.052359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:26.052359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.052359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:26.446293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:26.447168 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:26.447365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:26.448670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.448988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.449227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Traceback (most recent call last):
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.450097 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.451829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.452528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:26.453225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.453225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:26.454595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.455566 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.456590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:26.456590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.456590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:26.457547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:26.457967 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:26.458161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:26.458342 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:26.458989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.459362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.460221 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:26.461359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.462220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.462683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:26.463702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.464132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.464132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:26.464132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.464132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:26.464443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.464897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:26.465397 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:26.465708 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:26.466138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.466518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.467608 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:26.468354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.468768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.469336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:26.469675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.470053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.470053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:26.470053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.470053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.470336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:26.470839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:26.471383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:26.471665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:26.472021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.472280 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:26.472434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.473059 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.474182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.474545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:26.474930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.474930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:26.475263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:26.475778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:26.476317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:26.476317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:26.476317 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:31.008703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:31.009570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:31.010143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:31.011272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:31.046746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.047287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.048412 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.049290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.049699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:31.050146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.050146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:31.050474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.051041 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.051617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:31.051617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.051617 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:31.447185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:31.447912 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:31.448267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:31.449546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.449859 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.450174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.451031 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.452752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.453483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:31.454252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.454252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:31.454784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.455554 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.456120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:31.456120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.456120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:31.458024 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:31.459028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:31.459334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:31.459626 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:31.461837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.462103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.462857 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:31.463916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.464534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.464855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:31.465357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:31.465768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.466474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:31.467202 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:31.467665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:31.467665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:31.467665 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.467931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.468533 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:31.469450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.469897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.470635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:31.471052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.471499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.471499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:31.471499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.471499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.472266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:31.472898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:31.473540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:31.474201 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.474514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:31.474735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.475409 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.476501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.476966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:31.477456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.477456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:31.477711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:31.478338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:31.478981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:31.478981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:31.478981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:36.009709 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:36.010844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:36.010844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:36.012004 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:36.014927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:58:36.054035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.054035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.054791 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.055651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.056414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:36.057129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:36.058237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.059253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:36.059253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.059253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.059253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:36.059253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.059253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:36.447930 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:36.448823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:36.449072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:36.450661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.451090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.451342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] Traceback (most recent call last):
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.452411 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.454237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.454808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:36.455483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.455483 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-d0582eff-808f-4f57-afa6-88df93367079 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:36.456517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.457089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.457593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:36.457593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.457593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:36.458635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:36.459181 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:36.459404 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:36.459627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:36.460356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.460643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.461468 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:36.462941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.463661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.464597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:36.465251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.465874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.465874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:36.465874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.465874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:36.466108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.466831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:36.467661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:36.468218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:36.468218 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.468443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.469280 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:36.470656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.471351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.472111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:36.472628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.473323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.473323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:36.473323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.473323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.474102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:36.474726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:36.475469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:36.475902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.476298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:36.476548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.477434 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.478371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.478830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:36.479493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:36.479983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:36.480656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:36.480656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:36.480656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:36.480656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:36.480656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:36.480656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:41.010294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:41.011222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:41.011222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:41.012239 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:41.045022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.045498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.046652 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.047721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.049607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:41.050285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:41.051369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.052680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:41.052680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.052680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.052680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:41.052680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.052680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:41.479447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:41.480574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:41.480681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:41.480803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:41.481243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:41.482478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:41.482802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:41.483127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:41.483441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:41.484065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:41.484619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:41.484904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:41.485156 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:41.485517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.485948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:41.486208 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:41.486471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.486625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.487527 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.489011 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.489309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:41.489628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.489628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:41.489628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:41.490730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.491364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.491913 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:41.492869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:41.493449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.494318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:41.494751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.495470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.495470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:41.495470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.495470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:41.495470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:41.496469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.497438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:41.498756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:41.499655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:41.500212 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task update_ha_failover failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.500998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:41.502326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.502947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.503452 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.504188 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.018 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.504916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:41.505442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.505853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:41.506601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:41.507027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:41.507397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:41.507397 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:46.011224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:46.012047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:46.012253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:46.013621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:46.049643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.050353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.051553 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.052690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.053321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:46.053917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:46.054670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.055622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:46.055622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.055622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.055622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:46.055622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.055622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:46.481458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:46.520038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:46.522606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:46.522837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:46.523624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:46.524501 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:46.525580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:46.526136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:46.526377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:46.526541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:46.527355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.527355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.527755 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:46.527934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:46.528190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.529097 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:46.529282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:46.529505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Traceback (most recent call last):
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.529870 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.531304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.531848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:46.532405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.532405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:46.532405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-c32ed924-d5c3-412b-b0dd-336faf267fe6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:46.534127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.534727 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.535429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:46.535429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.535429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:46.535429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.536298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.536642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Traceback (most recent call last):
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.537249 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:46.539395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.540248 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:46.540794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.541644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:46.542092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.542513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.542513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:46.542513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.542513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:46.542858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:46.543376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-73a898ec-aee0-4b28-85d5-9fd28231f3b4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:46.544552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.545107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.545707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.546720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:46.547234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.547686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.547686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:46.547686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.547686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:46.547686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.549768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:46.550296 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Traceback (most recent call last):
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.550855 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] OVN maintenance task update_mac_aging_settings failed after 0.021 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.552183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.552573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:46.552946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.552946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-fd7a05b2-61b9-49e9-9db3-611878781de2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:46.553827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:46.554312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:46.554919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:46.554919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:46.554919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:51.012102 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:51.012791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:51.012997 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:51.014044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:51.046560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.047307 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.048261 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.049345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.050065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:51.050716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:51.051519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.052475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:51.052475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.052475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.052475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:51.052475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.052475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:51.520679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:51.522253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:51.522522 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:51.523532 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:51.523688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:51.583952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:51.583952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:51.583952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:51.583952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:51.584495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:51.585121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:51.585121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:51.585371 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:51.585705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:51.588151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.588717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:51.589425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.589425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.589848 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.590084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Traceback (most recent call last):
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.590596 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.068 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.592295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.592611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:51.592887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.592887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:51.592887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Traceback (most recent call last):
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.592969 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc5ac74f-637f-43a5-ba6d-cdce938ddf9f None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:51.594558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.594924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:51.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:51.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:51.596354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:51.596815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] OVN maintenance task set_fip_distributed_flag failed after 0.072 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.598375 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:51.598771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.599186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.599186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:51.599186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.599186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-a7f50a33-1fbc-43ba-8e87-3e0433fec9c9 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:51.599976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.600510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Traceback (most recent call last):
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:51.601094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:51.602559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] OVN maintenance task update_ha_failover failed after 0.018 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:51.603176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:51.603639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:51.603998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3db50c23-0f1f-41ce-8285-c0fb4a2012c7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:51.604386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.604952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Traceback (most recent call last):
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.605543 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] OVN maintenance task update_mac_aging_settings failed after 0.018 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.606439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.607107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:51.607466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.607466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-20067562-8109-411e-9564-fba10d0f5b3b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:51.607535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:51.608055 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:51.608595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:51.608595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:51.608595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:56.013034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:56.013949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:56.014203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:56.015929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:56.048599 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.049037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.050086 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.051046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.051697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:56.052405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:56.053346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.054377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:56.054377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.054377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.054377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:56.054377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.054377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:56.522047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:56.522671 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:56.522813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:56.524175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.524539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.524821 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.526065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.526965 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:56.527519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.528738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.529494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:56.530277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.530277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:56.530277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.530462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.531863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:56.532450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.532450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:56.532450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:58:56.532450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:56.532450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.532661 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:56.532874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.533573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:56.533721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.534125 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:56.535746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:56.536335 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.537820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:56.538268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.538741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.538741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:56.538741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.538741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:56.538741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:56.540359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.540882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:56.541474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.542104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:56.542636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.543054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:56.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:56.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.543715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:56.544264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.544850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.544850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:56.544850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.544850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:58:56.585341 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:58:56.585905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:58:56.586134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:58:56.586428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:58:56.586823 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.587214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:58:56.587568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Traceback (most recent call last):
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.627237 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] OVN maintenance task update_mac_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.628334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.628827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ce3a9c4e-3670-4473-9f6e-d6d321eea887 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:58:56.629095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:58:56.629405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:58:56.629752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:58:56.629752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:58:56.629752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:58:56.629752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:58:56.629752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:58:56.629752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:01.013939 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:01.014894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:01.014894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:01.015944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:01.053981 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.054591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.055737 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.056716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.057479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:01.058304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:01.059183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.060247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:01.060247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.060247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.060247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:01.060247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.060247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:01.522868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:01.523384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:01.523581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:01.524993 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.525480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.525761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.527198 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:01.527640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.530247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.530944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:01.531648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.531648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:01.531648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:01.531648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:01.531648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:01.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.534474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.535347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:01.535347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.535347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:01.535347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.535347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.535347 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:01.535700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:01.535892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:01.536151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.536625 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:01.538389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:01.538953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.540025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:01.540433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.540952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.540952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:01.540952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.540952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:01.540952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.542685 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:01.543279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.543908 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:01.544663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.545197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.546255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:01.546705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.547264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.547264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:01.547264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.547264 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:01.548672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.549337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.549337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:01.549337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.549337 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:01.586230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:01.586770 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:01.586956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:01.587331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:01.587984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.588470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:01.588730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.589978 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.590902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.591565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:01.592165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:01.592867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:01.593778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:01.593778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:01.593778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:01.593778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:01.593778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:01.593778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:06.014603 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:06.015478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:06.015710 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:06.017267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:06.062609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.062876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.063782 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.064814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.065470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:06.066111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:06.066921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.067917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:06.067917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.067917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.067917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:06.067917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.067917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:06.523525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:06.524345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:06.524578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:06.526139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.526517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.526761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.527884 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:06.528364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.530726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.531305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:06.531826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.531826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:06.531826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:06.533194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.533912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.535233 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.535810 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:06.536955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:06.537162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:06.537251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.537537 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:06.538787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:06.539703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.540310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:06.540970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.541687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.541687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:06.541687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.541687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:06.541687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.542408 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:06.543352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.543760 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.544149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:06.544449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.544449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.544697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:06.544913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.545208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.545208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:06.545208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.545208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.545859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:06.546272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.547049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.547049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:06.547049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.547049 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:06.586835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:06.587613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:06.587796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:06.588152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:06.589035 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.589516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:06.589753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Traceback (most recent call last):
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.590816 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.592763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.593298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:06.593676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.593676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-cc87c5fe-a9d0-4ab7-ac3d-85d89eea4c01 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:06.594797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:06.595357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:06.595971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:06.595971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:06.595971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:11.015384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:11.016114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:11.016285 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:11.017321 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:11.053674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.054245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.055460 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.056601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.057320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:11.058110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:11.058910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.060212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:11.060212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.060212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.060212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:11.060212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.060212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:11.524470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:11.525114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:11.525703 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:11.526829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.527372 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.527520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.528465 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.530339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.530873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:11.531369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.531369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:11.531369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:11.533555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.534246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.534806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:11.534806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.534806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:11.534806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:11.534806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:11.534806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:11.535044 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.535229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.535889 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:11.537495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:11.538250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.539763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:11.540240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.540714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.540714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:11.540714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.540714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:11.540714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.542169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:11.542728 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.543428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.543428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:11.543428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.543428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:11.543428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:11.543428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:11.543650 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.543900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.544811 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:11.546610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.547170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.547562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:11.548082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:11.548622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:11.549106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.549700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:11.587356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:11.587925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:11.588227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:11.588553 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:11.589460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.589836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:11.590147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Traceback (most recent call last):
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.591150 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.592713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.593544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:11.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-70791783-4515-4d07-95c0-dfec6f1532dd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:11.594980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:11.596099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:11.596949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:11.596949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:11.596949 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:16.016140 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:16.017183 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:16.017399 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:16.018837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:16.055111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.055992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.056870 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.057885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.058369 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:16.058786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:16.059341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.059849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:16.059849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.059849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.059849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:16.059849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.059849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:16.525167 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:16.526169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:16.526602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:16.528252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.528783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.529126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.530256 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.532093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.532678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:16.533266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.533266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:16.533677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.534382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:16.535172 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:16.535836 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.536245 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.537324 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:16.538678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:16.539163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.541021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:16.541429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.541809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.541809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:16.541809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.541809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:16.541809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.543937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:16.544564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:16.545624 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.546229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.547231 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:16.548266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.548926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.549371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:16.549881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:16.550531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:16.551211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.552177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:16.587975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:16.588852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:16.589117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:16.589478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:16.590600 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.590977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:16.591177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.592305 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.594174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.594706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:16.595265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.595265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:16.595391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:16.596105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:16.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:16.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:16.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:21.017375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:21.018396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:21.018396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:21.020076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:21.060908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.061259 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.061699 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.062733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.063495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:21.064199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:21.064976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.065963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:21.065963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.065963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.065963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:21.065963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.065963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:21.526292 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:21.526962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:21.527220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:21.528536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.528920 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.529228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.530242 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.532148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.532592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:21.532974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.532974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:21.533510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.534052 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.534641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:21.534641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.534641 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:21.534835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:21.535463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:21.535463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:21.535463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:21.536431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.536776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.537587 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:21.538575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.539850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:21.540289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.540718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.540718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:21.540718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.540718 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.541262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:21.541778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:21.542418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:21.542697 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.542946 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.543594 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:21.544484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.544975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.545378 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.545973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.546410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:21.546410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.546410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:21.547191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.547757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.547757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:21.547757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.547757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:21.588775 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:21.589490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:21.590180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:21.590180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:21.590543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.590923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.593686 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.594223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.594874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:21.595495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.595495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:21.595848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:21.596588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:21.597458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:21.597458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:21.597458 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:26.018288 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:26.019345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:26.019345 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:26.020138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:26.055867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.056360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.057439 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.058411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.059084 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:26.059742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:26.060563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.061569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:26.061569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.061569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.061569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:26.061569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.061569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:26.526919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:26.527256 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:26.527356 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:26.528687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.528984 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.529177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.530161 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.532237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.532528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:26.532784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.532784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:26.534118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.534580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.535586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:26.535586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.535586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:26.535586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:26.535959 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:26.536123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:26.536297 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:26.536914 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.537176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.537825 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:26.539489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.540133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.540825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:26.541222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.541608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.541608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:26.541608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.541608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.542530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:26.543062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:26.543580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:26.543812 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:26.544388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.544695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.545342 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:26.546144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.546618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.546996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.547461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.547852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:26.547852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.547852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.548095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:26.548608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.549167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.549167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:26.549167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.549167 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:26.589346 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:26.590369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:26.590544 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:26.590886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:26.591856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.592338 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:26.592589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.593599 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.595351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.595938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:26.596500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.596500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:26.596968 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:26.597813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:26.598622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:26.598622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:26.598622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:31.018846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:31.019573 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:31.019717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:31.020979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:31.058374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.058889 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.060129 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.061347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.061934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:31.062511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:31.063178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.063962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:31.063962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.063962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.063962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:31.063962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.063962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:31.527716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:31.528364 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:31.528564 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:31.529915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.530314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.530735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.531846 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.533213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.533875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:31.535520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:31.536424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:31.537147 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.537471 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:31.537895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.538405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.538743 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:31.539130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.539469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.539469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:31.539469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.539469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.539723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:31.540199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.540738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.540738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:31.540738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.540738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:31.543366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:31.543605 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:31.543708 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:31.543850 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:31.544192 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.544448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.544928 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:31.545656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.546107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.546555 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:31.546903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.547338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.547338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:31.547338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.547338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.547626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:31.548127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.548674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.548674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:31.548674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.548674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:31.590223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:31.591091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:31.591091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:31.591412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:31.591936 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.592319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:31.592535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.593367 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.594535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.594955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:31.595335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.595335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:31.595579 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:31.596060 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:31.596612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:31.596612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:31.596612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:36.019813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:36.020793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:36.020793 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:36.022123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:36.031826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=241 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 02:59:36.061220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.061759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.062892 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.063871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.064524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:36.065107 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:36.065812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.066698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:36.066698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.066698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.066698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:36.066698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.066698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:36.528616 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:36.529104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:36.529293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:36.530858 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.531366 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.531635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.532769 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.534507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.535147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:36.535677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.535677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:36.536115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.536815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:36.537590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:36.538176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.538522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.539390 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:36.540254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.540945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.541177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:36.541505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.541860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.541860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:36.541860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.541860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.542048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:36.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.543140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.543140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:36.543140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.543140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:36.543745 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:36.544065 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:36.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:36.544363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:36.544773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.545053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.545644 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:36.546456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.546905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.547563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:36.547891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.548287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.548287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:36.548287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.548287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.548667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:36.549258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.549890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.549890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:36.549890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.549890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:36.591150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:36.591807 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:36.592028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:36.592383 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:36.593108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.593567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:36.593834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.595287 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.596656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.597374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:36.598043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.598043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:36.598486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:36.599519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:36.600223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:36.600223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:36.600223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:41.020558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:41.021332 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:41.021441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:41.022335 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:41.054169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.054666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.055781 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.056690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.057350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:41.058112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:41.059126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.059990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:41.059990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.059990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.059990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:41.059990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.059990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:41.529442 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:41.530437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:41.530800 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:41.532353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.532739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.533003 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.534165 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.536731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.537409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:41.538101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.538101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:41.539166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.539996 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:41.540875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.541348 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.541679 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:41.542713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.543535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.543927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:41.544384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.544779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.544779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:41.544779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.544779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:41.545354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.545811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:41.546351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:41.546659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:41.547090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.547523 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.548390 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:41.549403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.549822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.550688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:41.551073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.551453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.551453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:41.551453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.551453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.552164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:41.552761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.553283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.553283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:41.553283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.553283 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:41.592339 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:41.593223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:41.593436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:41.593782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:41.594441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.594813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:41.595081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.595980 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.597724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.598342 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:41.598936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.598936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:41.599947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:41.600668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:41.601204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:41.601204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:41.601204 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:46.021305 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:46.022228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:46.022228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:46.023118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:46.056507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.056778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.057804 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.058950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.059676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:46.060432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:46.061330 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.062441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:46.062441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.062441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.062441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:46.062441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.062441 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:46.530284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:46.530629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:46.530773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:46.531847 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.532425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.532799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.533750 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.534919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.536023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.536779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:46.537814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.539136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.539136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:46.539136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.539136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:46.539827 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:46.540618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:46.540962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:46.541066 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:46.541585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.541798 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.542396 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:46.543514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.544216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.544705 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:46.545210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.545695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.545695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:46.545695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.545695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:46.545845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.546417 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:46.547110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:46.547474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:46.547474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:46.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.548765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.549679 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:46.550776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.551446 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.552102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:46.552556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.553089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.553089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:46.553089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.553089 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.553613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:46.554147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.554615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.554615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:46.554615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.554615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:46.593157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:46.593786 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:46.594115 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:46.594466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:46.595029 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.595480 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:46.595724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.596603 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.598428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.599053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:46.599634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.599634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:46.600409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:46.601241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:46.602163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:46.602163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:46.602163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:51.021962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:51.022893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:51.022893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:51.024370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:51.061257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.061820 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.063098 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.064913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.065604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:51.066359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.066359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:51.066797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.067683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.068713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:51.068713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.068713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:51.530856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:51.531314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:51.531314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:51.532177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.532495 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.532844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.533628 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.534524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.534904 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:51.535331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.535331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.535528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.536054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:51.536679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.536679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:51.540386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:51.541243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:51.541445 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:51.541655 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:51.542353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.542673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.543472 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:51.545484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.546095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.547254 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:51.547629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.547997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.547997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:51.547997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.547997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:51.549581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.550068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:51.550571 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:51.550885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:51.551355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.551589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.552359 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:51.553422 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.553826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.554359 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:51.554712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.555147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.555147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:51.555147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.555147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.555807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:51.556326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.556861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.556861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:51.556861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.556861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:51.593856 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:51.594700 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:51.594917 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:51.595320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:51.595866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.596275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:51.596496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.597473 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.599645 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.600252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:51.600835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.600835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:51.601198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:51.601948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:51.602793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:51.602793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:51.602793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:56.022794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:56.023977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:56.023977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:56.025215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:56.061500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.061782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.062655 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.063607 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.064290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:56.064960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:56.065802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.067078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:56.067078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.067078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.067078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:56.067078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.067078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:56.531591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:56.532059 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:56.532194 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:56.533652 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.534123 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.534430 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.535461 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.537277 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.538188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:56.538771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.538771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.538954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.539786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:56.540667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.540667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:56.540863 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:56.541674 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:56.541871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:56.542110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 02:59:56.542932 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.543177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.543983 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:56.545434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.546247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.546792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:56.547401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.548051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.548051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:56.548051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.548051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:56.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.549248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.550076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:56.550411 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:56.550996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:56.551122 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:56.551304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:56.551904 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.552329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.553165 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 02:59:56.554234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.554700 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.555197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:56.555562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.555966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.555966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:56.555966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.555966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.556322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:56.556874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.557439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.557439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:56.557439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.557439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 02:59:56.594601 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 02:59:56.595388 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 02:59:56.595555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 02:59:56.596154 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 02:59:56.596813 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.597306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 02:59:56.597571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.598639 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.600306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.600879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 02:59:56.601493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.601493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 02:59:56.602278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 02:59:56.603236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 02:59:56.604105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 02:59:56.604105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 02:59:56.604105 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:01.023554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:01.024407 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:01.024610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:01.025972 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:01.068054 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.068550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.069661 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.070683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.071394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:01.072154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:01.072970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.074131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:01.074131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.074131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.074131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:01.074131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.074131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:01.532455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:01.532832 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:01.532955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:01.534512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.534963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.535263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.536312 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.537525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.538191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.538709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:01.539197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.539734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.539734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:01.539734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.539734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:01.541385 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:01.541919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:01.542164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:01.542373 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:01.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.543275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.544096 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:01.545356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.545861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.546744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:01.547159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.547572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.547572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:01.547572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.547572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:01.548235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.548694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.549228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:01.550757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:01.551658 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:01.551786 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:01.551943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:01.552396 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.552638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.553226 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:01.554310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.554729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.555477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:01.555856 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.556257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.556257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:01.556257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.556257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.556521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:01.557039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.557522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.557522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:01.557522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.557522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:01.595516 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:01.596114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:01.596300 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:01.596621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:01.597278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.597640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:01.597882 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.598733 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.600614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.601268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:01.601851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.601851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:01.602262 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:01.603114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:01.603986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:01.603986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:01.603986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:06.024488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:06.025666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:06.025666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:06.027027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:06.066905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.067326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.068303 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.069409 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.069994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:06.070839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:06.071888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.072895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:06.072895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.072895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.072895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:06.072895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.072895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:06.533210 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:06.533539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:06.533539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:06.534724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.535117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.535354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.536238 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.537613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.538163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:06.538545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.538545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.539227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.539766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:06.540323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.540323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:06.541864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:06.542393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:06.542575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:06.542763 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:06.543962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.543962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.544673 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:06.545543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.546129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.546424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:06.546809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.547222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.547222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:06.547222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.547222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:06.547524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.548184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.548766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:06.551298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:06.551952 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:06.552106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:06.552266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:06.552758 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.553018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.553666 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:06.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.555263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.555940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:06.556532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.556973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.556973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:06.556973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.556973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.557332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:06.557845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.558480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.558480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:06.558480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.558480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:06.596203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:06.596651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:06.596919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:06.597264 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:06.597928 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.598258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:06.598427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.599176 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.600966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.601350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:06.601689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.601689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:06.602789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:06.603398 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:06.603937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:06.603937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:06.603937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:11.083450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:11.084443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:11.084443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:11.085749 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:11.120529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.120788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.121659 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.122603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.123630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:11.124615 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:11.125783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.127286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:11.127286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.127286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.127286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:11.127286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.127286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:11.534090 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:11.534765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:11.535165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:11.536519 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.536919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.537209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.538228 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.540195 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.540857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:11.541581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.541581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.542114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.543100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:11.544206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.544206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:11.544206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:11.544206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:11.544206 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:11.544497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:11.544875 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.545242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.546141 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:11.547249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.547881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.548286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:11.548651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.549164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.549164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:11.549164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.549164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:11.550478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.550956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.551509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:11.551705 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:11.552306 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:11.552492 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:11.552728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:11.553357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.553726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.554602 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:11.556764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.558321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:11.558747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.559288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.559288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:11.559288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.559288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.559692 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:11.560224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.560818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.560818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:11.560818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.560818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:11.596687 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:11.597462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:11.597633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:11.598042 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:11.599037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.599447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:11.599673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.600647 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.601716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.602424 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:11.603058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:11.603841 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:11.604840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:11.604840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:11.604840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:11.604840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:11.604840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:11.604840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:16.084282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:16.085427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:16.085427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:16.086410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:16.130036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.130610 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.132134 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.133654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.135276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:16.136109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:16.136984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.138246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:16.138246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.138246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.138246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:16.138246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.138246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:16.534883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:16.535712 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:16.535849 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:16.537435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.537887 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.538207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.539224 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.540993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.541964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.542623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:16.543448 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.544363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.544363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:16.544363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.544363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:16.544363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:16.544838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:16.544951 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:16.545165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:16.545938 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.546418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.547822 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:16.549992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.550558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.551470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:16.551863 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.552247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.552247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:16.552247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.552247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:16.553217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.553665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:16.554162 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:16.554469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:16.555067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.555439 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.556297 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:16.556938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.557382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.557642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:16.557966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:16.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:16.558780 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.559320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:16.597283 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:16.597781 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:16.597950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:16.598315 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:16.599139 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.599595 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:16.599853 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.600768 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.602319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.602982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:16.603626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:16.604385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:16.605383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:16.605383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:16.605383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:16.605383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:16.605383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:16.605383 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:21.084979 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:21.085925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:21.085925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:21.086765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:21.119023 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.119540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.120552 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.121860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.122501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:21.123197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:21.123997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.124932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:21.124932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.124932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.124932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:21.124932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.124932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:21.535971 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:21.536635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:21.536635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:21.538275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.538974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.538974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.539980 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.541668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.542469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.543356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:21.544428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:21.545970 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:21.546500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.546500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.547189 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:21.548527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.548930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.549903 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:21.550216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.550561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.550561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:21.550561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.550561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:21.551605 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.552139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.552678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:21.553991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:21.554435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:21.554613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:21.554829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:21.555451 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.556642 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:21.558030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.558565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.559484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:21.559971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.560580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.560580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:21.560580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.560580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.561265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:21.562100 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.562884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.562884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:21.562884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.562884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:21.598043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:21.598494 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:21.598629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:21.598935 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:21.599536 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.599883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:21.600192 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.600958 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.602434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.602986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:21.603392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.603392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:21.603463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:21.603938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:21.604467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:21.604467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:21.604467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:26.085761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:26.086826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:26.086826 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:26.088120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:26.123937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.123937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.124445 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.125205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.125847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:26.126533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:26.127269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.128163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:26.128163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.128163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.128163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:26.128163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.128163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:26.536606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:26.537228 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:26.537378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:26.538431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.538854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.539185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.540023 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.541544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.542350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.543207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:26.544191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.545255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.545255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:26.545255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.545255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:26.545255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:26.545722 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:26.545911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:26.546155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:26.546866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.547225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.548181 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:26.549609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.550592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.551139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:26.551797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.552597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.552597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:26.552597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.552597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:26.552810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.553647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:26.554569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:26.555026 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:26.555188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:26.555414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:26.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.556315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.557023 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:26.558392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.559155 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.559548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:26.559933 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.560343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.560343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:26.560343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.560343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.560648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:26.561292 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.561777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.561777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:26.561777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.561777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:26.598720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:26.599190 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:26.599365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:26.599648 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:26.600226 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.600610 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:26.600842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.601673 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.603278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.603901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:26.604493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.604493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:26.604936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:26.605719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:26.606675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:26.606675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:26.606675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:31.086525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:31.087499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:31.087499 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:31.087804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:31.127922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.128427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.129907 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.130817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.131496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:31.132062 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:31.132774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.133667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:31.133667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.133667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.133667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:31.133667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.133667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:31.537412 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:31.538148 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:31.538334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:31.539641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.539883 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.540172 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.541235 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.542898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.543648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:31.544431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.544431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.544661 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.545802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:31.546614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.546614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:31.546614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:31.546614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:31.546614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:31.546614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:31.546892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.546892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.547528 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:31.548472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.549125 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.549486 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:31.550056 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.550630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.550630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:31.550630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.550630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:31.550828 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.551496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.552261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:31.554991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:31.555443 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:31.555670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:31.556272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:31.556833 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.557265 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.558020 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:31.558935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.559562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.559983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:31.560485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:31.561081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:31.561738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.562560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:31.599133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:31.599940 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:31.600186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:31.600535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:31.601316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.601696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:31.601918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.602844 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.604953 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.605664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:31.606437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.606437 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:31.606872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:31.607665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:31.608326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:31.608326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:31.608326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:36.087378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:36.088308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:36.088512 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:36.089791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:36.091641 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=239 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 03:00:36.139175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.140091 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.141279 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.142038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.142709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:36.143343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:36.144316 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.145101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:36.145101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.145101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.145101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:36.145101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.145101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:36.538336 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:36.539180 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:36.539353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:36.540843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.541337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.541589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.542635 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.544536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.545215 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:36.545784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.545784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.546544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.547581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:36.548541 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.548849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.549432 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:36.550865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.551524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.552129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:36.552548 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.552927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.552927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:36.552927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.552927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.554019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:36.554535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.555216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.555216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:36.555216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.555216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:36.555530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:36.556304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:36.556916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:36.556916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:36.557423 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.557796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.559180 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:36.559916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:36.560721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:36.561463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:36.562096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:36.562619 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.563668 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:36.600859 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:36.601561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:36.601768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:36.602141 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:36.602872 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.603326 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:36.603573 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.604575 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.606228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.606877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:36.608467 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:36.609213 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:36.609852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:36.609852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:36.609852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:36.609852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:36.609852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:36.609852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:41.088394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:41.089149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:41.089259 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:41.090963 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:41.131441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.131861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.132956 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.134921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.135333 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:41.135717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.135717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:41.136995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.137926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.138737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:41.138737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.138737 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:41.539465 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:41.540314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:41.540460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:41.541504 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.541767 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.541994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.542876 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.543838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.544520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.545133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:41.545826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.546759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.546759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:41.546759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.546759 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:41.547377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:41.548169 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:41.548369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:41.548583 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:41.549215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.549440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.550159 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:41.551656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.552380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.553035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:41.553644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.554370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.554370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:41.554370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.554370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:41.554868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.555652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:41.556414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:41.556673 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:41.556854 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:41.557118 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:41.557739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.558135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.559154 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:41.560756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.561533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.562407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:41.563045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.563775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.563775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:41.563775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.563775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.564589 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:41.565468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.566494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.566494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:41.566494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.566494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:41.601900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:41.602852 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:41.603091 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:41.603429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:41.604334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.604751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:41.604981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.605898 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.607647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.608247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:41.608816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.608816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:41.609421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:41.610175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:41.610772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:41.610772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:41.610772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:46.089310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:46.090367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:46.090367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:46.091488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:46.128877 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.129194 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.130142 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.131126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.131776 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:46.132477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:46.133362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.134411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:46.134411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.134411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.134411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:46.134411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.134411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:46.540639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:46.542304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:46.542304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:46.542304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.542792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.542886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.543698 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.545131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.545740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.546396 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:46.547183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:46.548582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.549156 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.549234 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:46.549934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.550503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.550778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:46.551184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.551614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.551614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:46.551614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.551614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.551738 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:46.552350 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.552983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.552983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:46.552983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.552983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:46.556653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:46.557089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:46.557278 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:46.557485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:46.557871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.558118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.558672 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:46.559994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.560472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.561460 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:46.561810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.562242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.562242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:46.562242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.562242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.563279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:46.563805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.564344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.564344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:46.564344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.564344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:46.602659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:46.603629 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:46.603839 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:46.604717 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:46.605901 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.605901 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:46.605901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.606612 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.608323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.609154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:46.609698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.609698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:46.610028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:46.610750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:46.611587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:46.611587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:46.611587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:51.090165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:51.091316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:51.091316 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:51.092360 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:51.126363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.126884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.128014 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.129261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.129936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:51.130639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:51.131433 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.132374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:51.132374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.132374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.132374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:51.132374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.132374 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:51.541771 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:51.542302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:51.542455 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:51.543418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.543753 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.544048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.544867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.546249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.546844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:51.547403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.547403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.547871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.548734 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.549572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.550021 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:51.550371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.551207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:51.552182 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.552564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:51.552926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.553464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:51.557136 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:51.557410 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:51.557466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:51.557569 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:51.558435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.558897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.559854 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:51.560584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.561036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.561623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:51.561973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:51.562445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.562956 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:51.603561 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:51.604312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:51.604523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:51.604844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:51.605452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.605801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:51.606072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.606944 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.608761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.609219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:51.609576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.609576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:51.610510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:51.611051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:51.611784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:51.611784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:51.611784 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:56.091149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:56.092157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:56.092157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:56.093493 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:56.135713 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.136212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.137281 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.139241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.139864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:56.140688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.140688 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:56.142499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.143069 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.143754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:56.143754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.143754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:56.542602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:56.543258 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:56.543355 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:56.544698 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.545060 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.545323 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.546201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.547376 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.548073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.548729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:56.549553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:00:56.550598 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:56.551428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.551963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.552260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:56.552651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.553036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.553036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:56.553036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.553036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.553166 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:56.553648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.554208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.554208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:56.554208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.554208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:56.557636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:56.558481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:56.558679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:56.558908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:56.559867 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.560326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.561273 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:00:56.562875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.563557 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.564502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:56.564877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.565275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.565275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:56.565275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.565275 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.565807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:56.566380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.566873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.566873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:56.566873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.566873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:00:56.604394 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:00:56.604753 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:00:56.604960 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:00:56.605320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:00:56.606405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.606943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:00:56.607302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.608478 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.610077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.610498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:00:56.610868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.610868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:00:56.611600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:00:56.612153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:00:56.612735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:00:56.612735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:00:56.612735 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:01.091662 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:01.092633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:01.092742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:01.094074 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:01.131766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.132687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.134085 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.135219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.135788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:01.136463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.136463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:01.136753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.137363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.137909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:01.137909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.137909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:01.543462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:01.544117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:01.544503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:01.545576 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.545858 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.545958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.546634 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.548268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.548869 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:01.549503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.549503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.549868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.550846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:01.551761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.551761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:01.551761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:01.551761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:01.551761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:01.551761 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:01.552506 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.552849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.553643 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:01.554770 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.555765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.556321 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:01.556998 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.557698 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:01.558503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:01.559560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:01.560200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:01.560200 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.560200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.560766 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:01.561526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.562389 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.562706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.563096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:01.563572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:01.564134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.564839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:01.605036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:01.605838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:01.606070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:01.606429 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:01.607222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.607619 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:01.607878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.608879 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.610726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.611658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:01.613070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:01.614054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:01.615287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:01.615287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:01.615287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:01.615287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:01.615287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:01.615287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:06.092778 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:06.093803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:06.093803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:06.094602 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:06.133620 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.133937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.134655 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.135809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.136612 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:06.137320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:06.138223 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.139391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:06.139391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.139391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.139391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:06.139391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.139391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:06.544378 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:06.545052 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:06.545235 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:06.546738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.547209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.547596 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.548480 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.550339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.551164 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:06.551843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.551843 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.552036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.553251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.554211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.554716 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:06.555115 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.556137 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:06.556934 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.557695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:06.558403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.559604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:06.560253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:06.560253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:06.560253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:06.560253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:06.560414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.560624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.561240 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:06.562746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.563212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.564382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.564791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.565211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:06.565211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.565211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:06.566792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.567392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.567392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:06.567392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.567392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:06.605714 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:06.606170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:06.606304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:06.606487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:06.607038 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.607514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:06.607745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.608637 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.610136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.610814 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:06.611465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.611465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:06.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:06.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:06.613582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:06.613582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:06.613582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:11.093558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:11.095067 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:11.095254 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:11.096719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:11.127272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.127620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.128361 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.129075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.129724 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:11.130351 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:11.130809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.131364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:11.131364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.131364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.131364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:11.131364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.131364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:11.545337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:11.545866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:11.546086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:11.547558 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.547961 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.548272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.549266 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.550941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.551601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.552297 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:11.553112 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:11.554133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:11.554724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.555097 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.556123 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:11.557553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.558147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.559080 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:11.559491 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.559947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.559947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:11.559947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.559947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:11.559947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.561079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:11.561658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:11.562287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:11.562554 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.562616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.563169 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:11.564537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.565103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.565984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.566381 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.566870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:11.566870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.566870 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.567786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:11.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.568950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.568950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:11.568950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.568950 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:11.606549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:11.607088 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:11.607267 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:11.607580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:11.609676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.610217 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:11.610477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.611526 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.613186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:11.614710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.614710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:11.614910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:11.615804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:11.616726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:11.616726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:11.616726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:16.094441 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:16.095542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:16.095773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:16.097046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:16.134900 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.135191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.135867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.137030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.137649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:16.138325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:16.139154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.140128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:16.140128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.140128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.140128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:16.140128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.140128 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:16.545881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:16.546609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:16.546746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:16.548275 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.548640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.548876 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.550031 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.551824 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.552643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:16.553247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.553247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.553360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.554117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:16.554931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.554931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:16.554931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:16.554931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:16.554931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:16.554931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:16.555293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.555508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.556379 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:16.557729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.558673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:16.559526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.560218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:16.560954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.561683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:16.562253 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.562531 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:16.562531 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.562531 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.562531 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:16.562531 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.562531 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:16.563463 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.563941 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.565525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.566357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:16.566357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.566357 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.567726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:16.568246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.568918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.568918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:16.568918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.568918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:16.607491 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:16.607997 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:16.608204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:16.608618 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:16.609643 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.610203 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:16.610550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.611764 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.612952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.613833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:16.614690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:16.615775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:16.617325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:16.617325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:16.617325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:16.617325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:16.617325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:16.617325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:21.095243 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:21.096246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:21.096246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:21.096922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:21.138101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.138590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.139839 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.141239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.141627 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:21.142036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.142036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:21.142603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.143188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.143793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:21.143793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.143793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:21.546702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:21.547500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:21.547640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:21.549308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.549752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.550032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.551173 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.552792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.553197 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:21.553543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.553543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.554511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.554977 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:21.555525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.555525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:21.555525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:21.555741 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:21.555897 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:21.556273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:21.556769 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.557329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.558270 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:21.559712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.560471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.560840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:21.561372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.561892 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:21.562493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:21.563284 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.563840 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:21.564655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.565269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.565588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.566095 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:21.566626 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:21.567354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.568067 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:21.608128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:21.608679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:21.608874 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:21.609268 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:21.609794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.610166 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:21.610392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.611207 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.612861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.613421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:21.613960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.613960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:21.614613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:21.615400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:21.616242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:21.616242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:21.616242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:26.096639 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:26.097560 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:26.097720 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:26.099036 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:26.140039 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.140600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.141878 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.143103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.143674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:26.144305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:26.145042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.145928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:26.145928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.145928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.145928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:26.145928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.145928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:26.547695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:26.548466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:26.548623 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:26.549879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.550308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.550564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.551547 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.552797 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.553449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:26.554118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.554118 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.554356 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.555196 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:26.556086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.556086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:26.556086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:26.556086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:26.556086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:26.556086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:26.556370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.556492 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.556940 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:26.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.558696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:26.559462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.560092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:26.560690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.561807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:26.562870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:26.562870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:26.563082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:26.563188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:26.563659 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.563845 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.564334 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:26.565209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.565509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.566984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:26.567244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.567506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.567506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:26.567506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.567506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.568018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:26.568384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.568731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.568731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:26.568731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.568731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:26.608948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:26.609508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:26.609739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:26.610085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:26.611121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.611382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:26.611655 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.612717 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.614715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.615149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:26.615559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.615559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:26.616584 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:26.617108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:26.617669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:26.617669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:26.617669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:31.097568 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:31.098860 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:31.098945 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:31.100405 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:31.137893 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.138506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.139801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.140852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.141603 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:31.142175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:31.142884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.143811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:31.143811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.143811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.143811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:31.143811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.143811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:31.548581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:31.549370 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:31.549488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:31.550469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.550645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.550831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.551574 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.553044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.553720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:31.554528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.554528 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.554789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.555793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:31.556802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.556802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:31.556802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:31.556802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:31.556802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:31.556802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:31.557463 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.557756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.558623 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:31.560250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.560813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.561716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:31.562140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.562592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.562592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:31.562592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.562592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.563804 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:31.564379 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:31.565232 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:31.565586 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.565815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.566352 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:31.567110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.567762 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.568157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:31.568686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:31.569099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:31.569512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.570076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:31.609739 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:31.610676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:31.610884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:31.611325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:31.611911 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.612557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:31.612867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.613844 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.615279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.615865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:31.616582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.616582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:31.616935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:31.617679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:31.618775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:31.618775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:31.618775 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:36.099040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:36.100085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:36.100393 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:36.102323 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:36.104732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 03:01:36.136550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.137149 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.138189 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.139145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.139533 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:36.139984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.139984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:36.140099 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.140639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.141673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:36.141673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.141673 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:36.549314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:36.550502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:36.550502 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:36.551567 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.551953 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.552257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.553163 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.554360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.554720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:36.555217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.555217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.555440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.555908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:36.556413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.556413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:36.556413 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:36.556853 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:36.556987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:36.557170 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:36.557651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.557945 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.558657 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:36.559722 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.560487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.560741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:36.561114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.561501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.561501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:36.561501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.561501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.561744 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:36.563106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.564025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.564025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:36.564025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.564025 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:36.565185 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:36.565846 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:36.566176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:36.566470 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:36.567143 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.567413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.568214 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:36.569567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.569969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.571158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:36.571502 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.571905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.571905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:36.571905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.571905 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.572793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:36.573344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.573868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.573868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:36.573868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.573868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:36.610255 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:36.610947 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:36.611177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:36.611483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:36.612204 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.612427 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:36.612638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.613489 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.614833 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.615288 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:36.615639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.615639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:36.616471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:36.616997 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:36.617598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:36.617598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:36.617598 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:41.099866 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:41.100803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:41.100803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:41.101719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:41.138982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.139252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.140132 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.141057 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.141715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:41.142354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:41.143173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.144101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:41.144101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.144101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.144101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:41.144101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.144101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:41.550087 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:41.550827 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:41.551089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:41.552724 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.553205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.553513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.554582 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.556838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.557918 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:41.558629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.558629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.558922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.559811 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:41.560686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.560686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:41.560686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:41.560686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:41.560686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:41.560686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:41.561025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.561157 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.561835 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:41.563268 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.564740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:41.565699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.566251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.566754 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:41.567493 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:41.568353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.568736 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:41.569184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.569873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:41.570475 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:41.571043 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.571587 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.572443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:41.610865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:41.611575 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:41.611791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:41.612151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:41.613015 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.613291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:41.613583 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.614792 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.616322 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.616981 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:41.617648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.617648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:41.618629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:41.619459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:41.620320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:41.620320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:41.620320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:45.497178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:45.498271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:45.498271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:45.508211 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 13 03:01:45.508431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=107007) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 13 03:01:45.508431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Fixing resource 034f578d-aed5-407a-803d-deb792507504 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 03:01:45.544711 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-034f578d-aed5-407a-803d-deb792507504, resource={'id': '034f578d-aed5-407a-803d-deb792507504', 'name': 'private', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['43357193-c29e-4821-beec-551ae33a68c8', 'c34c04d1-5202-4926-a665-fa8b9cd2aa6f'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:26:31Z', 'updated_at': '2026-04-13T02:26:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19750}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.545193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-034f578d-aed5-407a-803d-deb792507504, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.545738 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-db0b0f73-0e32-4bf9-bf24-1366a8ba2a64, external_ids_update=None, columns={'tag': 19750}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.545881 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.546929 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Failed to fix resource 034f578d-aed5-407a-803d-deb792507504 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.547944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.548859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.549803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:45.549803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.549803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:45.549803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Fixing resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 03:01:45.579375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, resource={'id': 'e636fd1f-ea7c-4540-9032-93c72412bdfe', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['97900556-2eeb-4240-89bd-ed8258b9022d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:48Z', 'updated_at': '2026-04-13T02:30:49Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 9202}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.580177 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-e636fd1f-ea7c-4540-9032-93c72412bdfe, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8120b9cd-5d7c-4d4d-9a57-22a6c716f312, external_ids_update=None, columns={'tag': 9202}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.580652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.581931 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Failed to fix resource e636fd1f-ea7c-4540-9032-93c72412bdfe (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.583440 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.584075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.584765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:45.584765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.584765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:45.584765 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Fixing resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 03:01:45.621085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, resource={'id': '534a57c3-e4f0-4653-b521-48bccf54cc1e', 'name': 'admin_net', 'tenant_id': '6af9f010176b4c2789e19e116fa65ba0', 'project_id': '6af9f010176b4c2789e19e116fa65ba0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c04c79d3-5180-411b-a311-e3676c8b1a87'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-13T02:30:10Z', 'updated_at': '2026-04-13T02:30:11Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 19382}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.621422 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-534a57c3-e4f0-4653-b521-48bccf54cc1e, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.621902 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8ca97686-7da2-46a0-bfda-f15abb1e4ba1, external_ids_update=None, columns={'tag': 19382}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.622104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.623036 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Failed to fix resource 534a57c3-e4f0-4653-b521-48bccf54cc1e (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.624278 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.625549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:45.625549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.625549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:45.625549 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Fixing resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 03:01:45.661153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, resource={'id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'name': 'public', 'tenant_id': 'c169dfa6b3e840d8838c295312da70f8', 'project_id': 'c169dfa6b3e840d8838c295312da70f8', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'bfda765c-eb36-4dce-b302-ada4d43bd01a'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-13T02:26:38Z', 'updated_at': '2026-04-13T02:26:47Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.662157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.663138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-0d46232e-33db-40c1-91bf-c8d76fad6545, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.663465 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.664862 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Failed to fix resource 1b708029-1d8c-41a2-960e-97ac1e3ce3b5 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.666015 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.666339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.666606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:45.666606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.666606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:45.666606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Fixing resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 03:01:45.765459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, resource={'id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'project_id': 'f65616e5e4d24098b4a3fb745230ae46', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'enable_snat': True}, 'gw_port_id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-exgz1', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'external_fixed_ips': [{'ip_address': '172.24.5.236', 'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5'}, {'ip_address': '2001:db8::2c9', 'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 3, 'tenant_id': 'f65616e5e4d24098b4a3fb745230ae46'}, resource_type=routers, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.765985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-exgz1', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.766428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.766716 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "3ec69888-b9c9-4830-9d48-853dd81cf68b"')], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.766878 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.768480 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:45.768872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Unable to update router 530b037d-a47e-4f63-bd55-041139eeee2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Failed to fix resource 530b037d-a47e-4f63-bd55-041139eeee2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 267, in __exit__
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 13 03:01:45.770355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:45.770880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:45.771489 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Fixing resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports) at create/update {{(pid=107007) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 13 03:01:45.894488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, resource={'id': '3ec69888-b9c9-4830-9d48-853dd81cf68b', 'name': '', 'network_id': '1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'project_id': '', 'mac_address': 'fa:16:3e:92:a4:24', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '530b037d-a47e-4f63-bd55-041139eeee2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6d5e8d38-353b-44fa-825d-33890e3d6ff5', 'ip_address': '172.24.5.236'}, {'subnet_id': 'bfda765c-eb36-4dce-b302-ada4d43bd01a', 'ip_address': '2001:db8::2c9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000165409', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-13T02:30:50Z', 'updated_at': '2026-04-13T02:30:52Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.894879 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6d5e8d38-353b-44fa-825d-33890e3d6ff5 bfda765c-eb36-4dce-b302-ada4d43bd01a', 'neutron:network_name': 'neutron-1b708029-1d8c-41a2-960e-97ac1e3ce3b5', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-530b037d-a47e-4f63-bd55-041139eeee2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.236/24', '2001:db8::2c9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.895363 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=3ec69888-b9c9-4830-9d48-853dd81cf68b, lrouter_port=lrp-3ec69888-b9c9-4830-9d48-853dd81cf68b, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:45.895809 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Traceback (most recent call last):
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.896877 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] Maintenance task: Failed to fix resource 3ec69888-b9c9-4830-9d48-853dd81cf68b (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.898319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:45.898929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:45.899644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:45.899644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:45.899644 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:45.899644 np0000165409 neutron-ovn-maintenance-worker[107007]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ac5f6c0f-5a7a-4bbc-a1e1-6e87151af730 None None] OVN maintenance task check_for_inconsistencies finished in 0.401 seconds
Apr 13 03:01:46.100633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:46.101450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:46.101677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:46.103153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:46.137613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.138098 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.139111 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.140177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.140620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:46.141371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:46.142332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.143423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:46.143423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.143423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.143423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:46.143423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.143423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:46.550996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:46.551777 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:46.551777 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:46.553310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.553747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.554040 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.555119 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.556943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.557618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:46.558279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.558279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.558639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.559537 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:46.560496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.560496 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:46.560496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:46.560496 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:46.560737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:46.560737 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:46.561322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.561652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.562434 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:46.563908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.564509 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.565392 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:46.565768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:46.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.566214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.567423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:46.567965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.568637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.568637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:46.568637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.568637 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:46.568637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:46.568943 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:46.569309 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:46.569630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:46.570367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.570723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.571623 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:46.573506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.573973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.575170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:46.575532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.575943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.575943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:46.575943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.575943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.577319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:46.577795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.578299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.578299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:46.578299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.578299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:46.611467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:46.612120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:46.612420 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:46.612841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:46.613508 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.613937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:46.614216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.615033 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.616325 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.616670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:46.617014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.617014 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:46.617704 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:46.618170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:46.618662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:46.618662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:46.618662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:51.101835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:51.102885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:51.102885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:51.103822 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:51.145566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.145916 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.147130 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.148108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.148800 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:51.149490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:51.150300 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.151314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:51.151314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.151314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.151314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:51.151314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.151314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:51.551937 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:51.552357 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:51.552488 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:51.554110 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.554656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.554929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.555970 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.557764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.558380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:51.558917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.558917 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.559550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.560571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:51.561436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.561436 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:51.561436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:51.561436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:51.561436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:51.561436 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:51.561779 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.561849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.562500 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:51.563896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.564752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.565266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:51.565844 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.566470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.566470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:51.566470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.566470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.568522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:51.569071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:51.569656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:51.569908 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:51.571252 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.571827 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.572684 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:51.573562 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.574070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.574512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.574908 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:51.575868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:51.577074 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.577984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:51.611974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:51.612806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:51.612988 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:51.613430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:51.614022 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.614550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:51.614654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.615363 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.616545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.616984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:51.617366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.617366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:51.617606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:51.618169 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:51.618826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:51.618826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:51.618826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:56.102630 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:56.103622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:56.103622 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:56.105010 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:56.142948 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.143810 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.145020 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.146331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.147032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:56.147679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:56.148466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.149407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:56.149407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.149407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.149407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:56.149407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.149407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:56.552633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:56.552994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:56.553121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:56.554165 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.554434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.554662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.555474 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.556803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.557530 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.558281 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:56.559126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.560174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.560174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:56.560174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.560174 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:56.560570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:56.560732 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:56.560817 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:56.560884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:01:56.561145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.561319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.561711 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:56.562423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.562995 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.563298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:56.563690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.564124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.564124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:56.564124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.564124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.564248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:56.564731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.565318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.565318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:56.565318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.565318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:56.569449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:56.570061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:56.570197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:56.570354 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:56.571205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.571513 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.572209 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:01:56.573237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.573842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.574407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:56.574789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.575345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.575345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:56.575345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.575345 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.575618 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:56.576136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.576683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.576683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:56.576683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.576683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:01:56.612679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:01:56.613478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:01:56.613721 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:01:56.614163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:01:56.615271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.615715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:01:56.615964 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.617024 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.618907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.619568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:01:56.620176 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:01:56.620919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:01:56.621792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:01:56.621792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:01:56.621792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:01:56.621792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:01:56.621792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:01:56.621792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:01.103382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:01.104251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:01.104566 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:01.106690 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:01.144483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.144866 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.145429 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.146449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.147030 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:01.147447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.147447 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:01.147550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.148559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.149544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:01.149544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.149544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:01.553333 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:01.553649 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:01.553740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:01.555418 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.555835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.556079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.557034 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.558822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.559542 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:01.560162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.560162 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.560656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.561895 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.563178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.563510 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:01.564779 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.565561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.566522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:01.567299 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.567697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.567697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:01.567697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.567697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.567897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:01.568309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.568691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.568691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:01.568691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.568691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:01.569803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:01.570082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:01.570224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:01.570365 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:01.570996 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.571468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.572294 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:01.573022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.573482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.574225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:01.574636 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:01.575146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:01.575581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.576129 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:01.613449 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:01.614368 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:01.614535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:01.614871 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:01.615452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.615990 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:01.615990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.616708 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.618406 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.619147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:01.619697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.619697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:01.620047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:01.620951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:01.622329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:01.622329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:01.622329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:06.104214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:06.109249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:06.109249 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:06.109555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:06.147539 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.147791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.148550 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.149749 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.150423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:06.150994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:06.151923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.152666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:06.152666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.152666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.152666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:06.152666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.152666 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:06.553965 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:06.554196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:06.554272 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:06.555523 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.555941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.556267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.557380 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.558455 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.559201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.560145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:06.560914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.562153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.562153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:06.562153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.562153 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:06.562153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:06.562742 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:06.562825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:06.562889 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:06.563462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.563801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.564418 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:06.565205 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.565808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:06.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.566825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:06.567629 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.568387 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:06.570296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:06.571233 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:06.571322 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:06.571490 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:06.572050 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.572328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.572992 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:06.573788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.574377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.574683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:06.575046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:06.575571 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:06.575990 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.577130 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:06.614501 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:06.615266 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:06.615382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:06.615592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:06.616507 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.616913 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:06.617165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.619338 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.620303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.620799 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:06.621525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:06.622044 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:06.623093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:06.623093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:06.623093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:06.623093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:06.623093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:06.623093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:11.138944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:11.138944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:11.138944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:11.138944 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:11.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.162120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.162873 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.163736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.164258 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:11.164864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:11.165702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.166434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:11.166434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.166434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.166434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:11.166434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.166434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:11.554702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:11.555487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:11.555487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:11.556579 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.557816 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.557816 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.558053 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.558778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.559318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:11.559826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.559826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.560355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.560913 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:11.561581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.561581 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:11.563607 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:11.564271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:11.564271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:11.564271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:11.564743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.565111 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.565667 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:11.566715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.567282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.569547 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:11.569955 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.570372 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:11.570778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:11.571308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:11.571841 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:11.571975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:11.572163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:11.573179 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.573179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.574082 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:11.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.576214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.578978 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:11.579362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.579736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.579736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:11.579736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.579736 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.581019 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:11.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.582132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.582132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:11.582132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.582132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:11.618597 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:11.619946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:11.619946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:11.619946 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:11.620796 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.624540 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:11.624864 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.625825 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.627366 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.630225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:11.630560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:11.630944 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:11.631473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:11.631473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:11.631473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:11.631473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:11.631473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:11.631473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:16.110612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:16.111350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:16.111350 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:16.111783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:16.148376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.148678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:16.150897 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:16.151887 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.152712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:16.153651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.154812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:16.555534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:16.556155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:16.556257 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:16.557424 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.557811 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.557986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.558789 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.559898 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.560177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:16.560414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.560414 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.561363 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.561729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:16.562144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.562144 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:16.564916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:16.565975 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:16.566076 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:16.566215 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:16.566627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.567071 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.567545 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:16.568473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.569022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.569295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:16.569693 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.570134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.570134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:16.570134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.570134 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.570368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:16.570902 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.571461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.571461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:16.571461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.571461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:16.571591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:16.572040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:16.572121 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:16.572260 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:16.572651 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.572891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.573391 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:16.574214 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.574907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.575103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:16.575343 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.575600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.575600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:16.575600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.575600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.575915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:16.576304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.576679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.576679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:16.576679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.576679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:16.619446 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:16.619869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:16.620009 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:16.620244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:16.621018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.621290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:16.621451 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.622201 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.623695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.623987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:16.624269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.624269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:16.624771 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:16.625126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:16.625507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:16.625507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:16.625507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:21.111296 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:21.112271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:21.112572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:21.114663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:21.157434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.157758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.158243 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.158993 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.159708 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:21.160461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:21.161352 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.162285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:21.162285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.162285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.162285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:21.162285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.162285 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:21.556447 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:21.556919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:21.557093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:21.558640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.559127 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.559327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.560280 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.561847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.562442 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:21.562966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.562966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.563473 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.564309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:21.565127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.565127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:21.566270 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:21.566747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:21.566916 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:21.567104 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:21.568133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.568560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.570269 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:21.571947 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.572595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.574168 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:21.574613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.574974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.574974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:21.574974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.574974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:21.574974 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.576520 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:21.577094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.577664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.577664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:21.577664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.577664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:21.577664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:21.577664 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:21.577880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:21.578500 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.578852 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.579535 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:21.580341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.580808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.581260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:21.581786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.582456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.582456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:21.582456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.582456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.582657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:21.583315 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.583932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.583932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:21.583932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.583932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:21.620302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:21.621069 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:21.621391 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:21.621776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:21.622481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.622925 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:21.623241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.624109 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.625248 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.625982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:21.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.626609 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:21.626888 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:21.627469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:21.628047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:21.628047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:21.628047 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:26.112142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:26.113153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:26.113259 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:26.114525 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:26.158106 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.158314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.159279 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.160791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.161825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:26.162550 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:26.163318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.164227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:26.164227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.164227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.164227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:26.164227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.164227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:26.557319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:26.557776 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:26.558031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:26.559459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.559799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.560108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.561077 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.562231 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.562826 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.563783 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:26.564606 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.565732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.565732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:26.565732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.565732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:26.566685 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:26.567271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:26.567384 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:26.567454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:26.567891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.568563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.568858 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:26.569940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.570682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.571289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:26.571932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.572536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.572536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:26.572536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.572536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.572781 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:26.573338 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.573976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.573976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:26.573976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.573976 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:26.577403 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:26.578031 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:26.578251 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:26.578452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:26.579152 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.579625 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.580519 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:26.581499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.581994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.582560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:26.583088 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.583466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.583466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:26.583466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.583466 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.583719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:26.584267 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.585026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.585026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:26.585026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.585026 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:26.621182 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:26.621881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:26.622025 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:26.622271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:26.622788 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.623271 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:26.623529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.624635 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.626370 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.627096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:26.627613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.627613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:26.628132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:26.628848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:26.629582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:26.629582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:26.629582 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:31.112869 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:31.113759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:31.113759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:31.114331 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:31.154034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.154371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.155257 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.156599 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.156872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:31.157143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.157143 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:31.157764 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.158246 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.158601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:31.158601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.158601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:31.557934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:31.558517 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:31.558574 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:31.560040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.560696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.560979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.561823 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.562912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.563224 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:31.563556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.563556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.563706 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.564086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:31.564468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.564468 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:31.567273 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:31.567794 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:31.567868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:31.567958 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:31.568466 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.568758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.569336 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:31.570291 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.570648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.571179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:31.571503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.571787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.571787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:31.571787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.571787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.572720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:31.573252 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.573838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.573838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:31.573838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.573838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:31.577899 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:31.578663 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:31.578844 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:31.579089 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:31.579666 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.580188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.583426 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:31.584479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.585217 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.585490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:31.585817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.586229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.586229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:31.586229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.586229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.586590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:31.587127 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.587691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.587691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:31.587691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.587691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:31.621941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:31.622634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:31.622804 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:31.623157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:31.624244 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.624764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:31.625103 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.626068 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.627952 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.629459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:31.630072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.630072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:31.631558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:31.632386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:31.633538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:31.633538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:31.633538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:36.113467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:36.114274 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:36.114621 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:36.116193 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:36.117954 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=434 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 03:02:36.152637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.153145 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.154391 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.156339 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.157038 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:36.157588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.157588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:36.158320 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.159891 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.160662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:36.160662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.160662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:36.558821 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:36.560924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:36.560924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:36.561582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.561888 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.562139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:36.565184 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.565774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:36.566332 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:36.566822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.567459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:36.567751 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:36.568677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:36.568677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:36.569706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:36.570340 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.570660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.571650 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:36.572694 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.574253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:36.575092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.575462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.577327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:36.578429 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.578973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.578973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:36.578973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.578973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:36.578973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:36.579642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:36.579642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:36.579642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:36.579985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.580191 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.580701 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:36.581796 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:36.583136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:36.584355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.584707 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:36.585970 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.586494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.586494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:36.586494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.586494 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:36.622819 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:36.623706 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:36.623843 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:36.624100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:36.624803 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.625369 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:36.625687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.626697 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.627859 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.628226 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:36.628565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.628565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:36.629532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:36.630623 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:36.631588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:36.631588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:36.631588 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:41.114294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:41.115095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:41.115095 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:41.115581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:41.154116 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.154116 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.154564 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.155621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.156087 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:41.156505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.156505 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:41.156753 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.157289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.157830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:41.157830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.157830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:41.560704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:41.562484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:41.562484 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:41.563343 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.563829 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.564165 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.565110 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.567654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.568141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.568543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:41.569051 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:41.570552 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.570867 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:41.572045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.572620 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.573514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:41.573802 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.574198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.574198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:41.574198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.574198 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.575310 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:41.575690 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.576079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.576079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:41.576079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.576079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:41.579171 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:41.579894 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:41.580108 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:41.580223 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:41.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.581054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.581591 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:41.582676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.583200 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.584139 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:41.584529 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.584954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.584954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:41.584954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.584954 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.586263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:41.586969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.587600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.587600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:41.587600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.587600 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:41.623868 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:41.624828 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:41.625071 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:41.625400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:41.626485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.627021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:41.627328 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Traceback (most recent call last):
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.629665 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.631261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.631808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:41.632541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.632541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-ceab9f7a-aca2-4559-9fc7-88b7b16fda84 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:41.632873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:41.633793 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:41.634850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:41.634850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:41.634850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:46.115043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:46.116261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:46.116261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:46.116950 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:46.154744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.155106 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.155795 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.157150 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.157592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:46.157969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.157969 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:46.158341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.158854 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.159470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:46.159470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.159470 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:46.561929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:46.562452 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:46.562590 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:46.563870 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.564302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.564561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.565660 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.566901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.567274 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:46.567539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.567539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:46.570148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.570848 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:46.571425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.571831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.572729 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:46.573484 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:46.574563 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.575298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.575910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:46.577569 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.578386 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.579101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:46.580546 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:46.581609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:46.581609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:46.581609 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:46.582103 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.582660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.583772 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:46.584751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.585403 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.585720 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:46.586135 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.586560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.586560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:46.586560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.586560 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.586901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.587987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.587987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:46.587987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.587987 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:46.624555 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:46.625310 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:46.625431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:46.625587 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:46.626159 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.626464 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:46.626643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.627652 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.628750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:46.629456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:46.630075 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:46.630932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:46.631752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:46.631752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:46.631752 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:51.115929 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:51.116991 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:51.117155 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:51.118287 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:51.159715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.160514 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.161658 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.162556 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.163251 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:51.163838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:51.164522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.165500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:51.165500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.165500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.165500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:51.165500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.165500 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:51.562891 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:51.563614 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:51.563740 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:51.564962 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.565518 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.565936 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.567202 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.568394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.568813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:51.569243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.569243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:51.569495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.570183 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.570885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:51.570885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.570885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:51.570885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:51.571334 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:51.571521 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:51.571718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:51.572224 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.572453 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.573078 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:51.574314 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.575045 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.575517 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:51.576093 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.576656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.576656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:51.576656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.576656 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:51.577419 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.577979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.578539 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:51.581093 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:51.581458 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:51.581613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:51.581835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:51.582348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.582714 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.583640 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:51.584367 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.585068 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.585479 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:51.586033 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:51.586608 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:51.587219 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.587923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:51.625400 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:51.625964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:51.626174 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:51.626430 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:51.627188 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.627515 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:51.627674 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.628541 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.630160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:51.630807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.631302 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:51.631768 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:51.632294 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:51.632911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:51.632911 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:56.117348 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:56.118351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:56.118533 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:56.119640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:56.164799 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.165244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.166076 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.167234 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.168199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:56.168862 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:56.169725 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.170857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:56.170857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.170857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.170857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:56.170857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.170857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:56.563931 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:56.564580 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:56.564757 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:56.566344 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.566728 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.566963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.567846 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.569362 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.569825 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:56.570211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.570211 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:56.570420 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.570894 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:56.571462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:02:56.571783 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.572120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.572926 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:56.573847 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.574382 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.574747 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:56.575133 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.575576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.575576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:56.575576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.575576 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:56.575907 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.576428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.576986 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:56.581460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:56.581686 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:56.581797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:56.581949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:56.582514 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.582900 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.583716 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:02:56.584649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.585136 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.586131 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:56.586488 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.586871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.586871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:56.586871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.586871 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.587861 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:56.588450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.588974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.588974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:56.588974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.588974 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:02:56.626046 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:02:56.626627 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:02:56.626752 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:02:56.626886 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:02:56.627450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.627880 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:02:56.628225 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.629094 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.629890 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:02:56.630506 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:02:56.631083 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:02:56.631880 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:02:56.632786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:02:56.632786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:02:56.632786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:01.118403 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:01.119635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:01.119635 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:01.120782 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:01.159142 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.159142 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.159551 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.160710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.161401 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:01.162201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:01.162886 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.163663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:01.163663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.163663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.163663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:01.163663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.163663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:01.564837 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:01.565838 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:01.566505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:01.567428 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.567756 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.567962 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.569035 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.570516 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.571188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:01.571742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.571742 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:01.572286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.573072 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:01.573915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.574353 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.574709 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:01.576202 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.576739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.577536 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:01.577972 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.578459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.578459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:01.578459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.578459 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:01.579190 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.579712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.580290 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:01.581797 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:01.582064 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:01.582176 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:01.582311 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:01.582634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.582808 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.583531 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:01.584327 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.585186 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:01.585572 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.586031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.586031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:01.586031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.586031 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.586221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:01.586740 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.587395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.587395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:01.587395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.587395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:01.626715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:01.627377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:01.627675 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:01.627764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:01.628434 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.629061 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:01.629061 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.629819 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.630940 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:01.631635 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:01.632037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:01.632564 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:01.633270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:01.633270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:01.633270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:06.119238 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:06.119982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:06.119982 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:06.121320 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:06.156295 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.156677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.157585 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.158526 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.159286 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:06.160050 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:06.160912 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.162048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:06.162048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.162048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.162048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:06.162048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.162048 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:06.565905 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:06.567020 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:06.567290 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:06.568291 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.568578 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.568807 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.570329 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.571628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.572263 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:06.572829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.572829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:06.573154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.573709 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.574237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:06.574237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.574237 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:06.574237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:06.574237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:06.574237 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:06.574454 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:06.574864 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.575094 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:06.577295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.577765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.578660 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:06.579021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.579368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.579368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:06.579368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.579368 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:06.580319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.580786 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.581270 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:06.582138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:06.582301 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:06.582382 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:06.582483 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:06.582978 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.583487 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.584249 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:06.584818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.585260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.585518 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:06.585838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:06.586235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:06.586647 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.587178 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:06.627353 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:06.627636 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:06.627815 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:06.628086 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:06.628670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.629092 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:06.629336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.630229 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.631715 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:06.632410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.632943 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:06.633648 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:06.634384 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:06.634901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:06.634901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:11.121157 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:11.123450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:11.123450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:11.123450 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:11.164114 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.164490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.166124 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.167242 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.167512 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:11.167756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.167756 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:11.169079 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.169385 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.169726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:11.169726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.169726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:11.566985 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:11.567681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:11.567681 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:11.570304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.570304 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.570304 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.572801 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.574646 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.575247 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:11.575602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.575602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:11.576672 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.577126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.577592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:11.577592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.577592 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:11.578390 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:11.578835 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:11.578987 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:11.579153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:11.580591 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.580591 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.580805 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:11.581787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.582341 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.582857 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:11.583639 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.583980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.583980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:11.583980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.583980 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:11.584147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.584559 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:11.585021 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:11.585308 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:11.586469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.586469 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.586680 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:11.587782 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.588503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.588885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:11.589405 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.590042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.590042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:11.590042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.590042 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.590120 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:11.590806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.591306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.591306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:11.591306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.591306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:11.627977 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:11.628633 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:11.628719 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:11.628881 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:11.631199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.631199 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:11.631199 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.631758 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.632185 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:11.632805 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:11.633309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:11.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:11.634293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:11.634293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:11.634293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:16.122547 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:16.123435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:16.123435 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:16.124319 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:16.162030 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.162597 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.163719 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.165086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.166260 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:16.166932 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:16.167712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.168601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:16.168601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.168601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.168601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:16.168601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.168601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:16.567766 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:16.568294 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:16.568498 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:16.569980 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.570389 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.570664 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.571675 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.573686 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.574255 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:16.574732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.574732 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:16.575551 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.576147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.576719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:16.576719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.576719 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:16.578892 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:16.579282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:16.579474 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:16.579679 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:16.580358 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.580679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.581544 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:16.582677 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.583141 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.583489 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:16.584092 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:16.584713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:16.586293 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:16.586973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:16.586973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:16.586973 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.586973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.587407 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:16.588445 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.589193 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.589624 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:16.590434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:16.591239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:16.591927 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.592818 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:16.628581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:16.629153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:16.629361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:16.629668 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:16.630186 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.630531 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:16.630788 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Traceback (most recent call last):
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.631561 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.632979 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.633683 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:16.634331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.634331 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f3b2dda4-42a2-41c4-8fc0-0a0f9ae0f588 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:16.634948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:16.635729 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:16.636640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:16.636640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:16.636640 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:21.123351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:21.124376 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:21.124503 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:21.125994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:21.166927 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.167388 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.168464 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.169649 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.170434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:21.171177 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:21.172257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.173498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:21.173498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.173498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.173498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:21.173498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.173498 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:21.568585 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:21.569526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:21.569772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:21.571034 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.571367 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.571552 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.572477 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.574305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.574960 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:21.575558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.575558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:21.576063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.576803 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.577638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:21.577638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.577638 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:21.579214 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:21.579530 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:21.579653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:21.579791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:21.580175 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.580400 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.580874 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:21.581896 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.582410 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.583411 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:21.583790 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.584203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.584203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:21.584203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.584203 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:21.585426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.585909 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:21.586471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:21.586831 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:21.587085 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:21.587337 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:21.587809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.588059 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.588654 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:21.590261 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.590723 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.591829 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:21.592188 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.592614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.592614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:21.592614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.592614 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.593687 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:21.594326 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.594872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.594872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:21.594872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.594872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:21.629100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:21.629689 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:21.629857 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:21.630231 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:21.630702 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.630934 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:21.631256 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.631726 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.633371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:21.634037 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.634616 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:21.635220 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:21.636081 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:21.636919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:21.636919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:26.124302 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:26.125227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:26.125227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:26.126361 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:26.156695 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.157132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.157988 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.158712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.159354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:26.159973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:26.160713 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.161593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:26.161593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.161593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.161593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:26.161593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.161593 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:26.569628 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:26.570050 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:26.570146 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:26.571572 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.572100 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.572390 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.573495 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.575347 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.575819 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:26.576208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.576208 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:26.577210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.577678 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.578243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:26.578243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.578243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:26.579606 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:26.580134 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:26.580261 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:26.580392 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:26.580915 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.581295 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.582066 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:26.582873 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.583717 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:26.584235 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.584590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.584590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:26.584590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.584590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:26.584846 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.585311 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.585745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:26.586469 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:26.586744 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:26.586865 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:26.587017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:26.587529 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.587961 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.588745 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:26.589538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.589951 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.590412 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:26.590699 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.591114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.591114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:26.591114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.591114 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.591456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:26.591973 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.592511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.592511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:26.592511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.592511 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:26.629672 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:26.630282 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:26.630377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:26.630577 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:26.631374 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.631802 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:26.632266 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.633147 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.634238 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:26.634746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.635110 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:26.635243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:26.635763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:26.636309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:26.636309 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:31.125230 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:31.126386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:31.126386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:31.127701 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:31.163120 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.163622 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.164812 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.165982 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.166765 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:31.167524 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:31.168438 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.169485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:31.169485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.169485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.169485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:31.169485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.169485 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:31.570535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:31.571581 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:31.571801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:31.573043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.573612 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.573948 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.577501 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.578868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.579565 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:31.580244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.580244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:31.580543 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.581289 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:31.582250 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:31.582773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.582994 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.583651 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:31.585123 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.585628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.586540 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:31.586919 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.587312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.587312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:31.587312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.587312 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:31.587868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.588306 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:31.588806 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:31.589145 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:31.589625 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.590146 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.590992 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:31.591851 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.592279 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.592910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:31.593272 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.593684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.593684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:31.593684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.593684 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.594078 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:31.594602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.595148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.595148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:31.595148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.595148 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:31.630478 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:31.631149 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:31.631312 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:31.631550 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:31.632313 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.632759 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:31.633058 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.634354 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.636161 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:31.637104 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:31.637868 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:31.638924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:31.640070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:31.640070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:31.640070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:36.126163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:36.127111 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:36.127220 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:36.128885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:36.131037 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG dbcounter [-] [107007] Writing DB stats neutron:SELECT=240 {{(pid=107007) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 13 03:03:36.169070 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.169480 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.170340 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.171427 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.172221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:36.172958 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:36.173712 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.174613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:36.174613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.174613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.174613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:36.174613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.174613 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:36.571704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:36.572505 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:36.572750 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:36.574676 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.575151 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.575435 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.576568 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.578421 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.579227 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:36.579910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.579910 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:36.580212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.581457 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:36.582792 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.583336 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.583730 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:36.584346 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.584815 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.585124 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:36.585499 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:36.585867 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.586298 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.586938 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:36.588425 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:36.588684 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:36.588773 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:36.588949 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:36.589414 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.589777 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Traceback (most recent call last):
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.590585 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:36.591471 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.591813 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.592426 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:36.592733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.593096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.593096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:36.593096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.593096 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.593621 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:36.594046 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.594461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.594461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:36.594461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.594461 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:36.631317 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:36.631923 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:36.632135 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:36.632460 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:36.633153 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.633542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:36.633792 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.634669 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.636241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:36.637070 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.637695 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:36.637761 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:36.638604 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:36.639544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:36.639544 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:41.127117 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:41.128043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:41.128298 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:41.129747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:41.165768 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.165957 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.167035 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.168023 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.168413 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:41.168875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.168875 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:41.169117 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.169519 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.169885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:41.169885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.169885 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:41.572526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:41.573534 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:41.573746 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:41.575209 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.575688 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.575942 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.577022 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.578721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.579335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:41.580032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.580032 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:41.580482 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.581276 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.582138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:41.582138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.582138 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:41.582138 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:41.582637 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:41.582824 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:41.583028 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:41.583647 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.583975 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.584789 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:41.586170 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.586602 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.587643 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:41.588027 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.588546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.588546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:41.588546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.588546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:41.589423 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.589872 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:41.590377 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:41.590656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:41.590656 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:41.591150 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.591521 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Traceback (most recent call last):
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.592319 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:41.593132 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.593586 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.594017 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:41.594329 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.594731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.594731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:41.594731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.594731 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.595175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:41.595679 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.596210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.596210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:41.596210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.596210 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:41.631994 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:41.632670 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:41.632885 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:41.633279 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:41.633787 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.634195 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:41.634434 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.635228 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.636922 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:41.637893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.638831 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:41.639175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:41.640063 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:41.640553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:41.640553 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:46.128040 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:46.128924 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:46.129164 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:46.130642 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:46.175069 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.175696 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.176923 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.178508 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.179239 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:46.179915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.179915 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:46.180201 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.181236 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.182395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:46.182395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.182395 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:46.573487 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:46.574101 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:46.574387 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:46.575801 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.576262 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.576522 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.577580 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.579371 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.579984 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:46.580631 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:46.581355 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.582318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:46.582318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.582318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.582318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:46.582318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.582318 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:46.582592 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:46.583471 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:46.583808 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:46.584227 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:46.584677 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.584983 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.585822 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:46.587472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.587924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.588906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:46.589303 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.589681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.589681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:46.589681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.589681 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:46.590733 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.591282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:46.591791 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:46.592128 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:46.592485 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.592860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Traceback (most recent call last):
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.593691 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:46.594527 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.594926 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.595432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:46.595769 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.596240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.596240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:46.596240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.596240 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-97d8cadf-5ce3-4e96-951b-50060d7a3f48 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.597028 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:46.597601 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.598101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.598101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:46.598101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.598101 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:46.632634 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:46.633222 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:46.633375 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:46.633743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:46.634401 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.634809 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:46.635102 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.635966 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.637924 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:46.638914 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.639850 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:46.640222 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:46.641158 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:46.641657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:46.641657 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:51.128906 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:51.129696 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:51.130133 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:51.130825 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:51.170715 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.170988 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.171849 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.173122 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.173663 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:51.174054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.174054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:51.174282 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.174921 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.175503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:51.175503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.175503 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:51.574497 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:51.575246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:51.575481 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:51.576989 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.577467 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.577739 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.578730 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.579930 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.580590 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:51.581257 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:51.582108 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:51.583077 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:51.583408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:51.583408 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:51.583545 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:51.584027 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.584271 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.584929 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:51.585785 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.586244 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.587662 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:51.588324 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:51.588778 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.589212 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.589702 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:51.591386 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:51.591922 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:51.592197 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:51.592439 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:51.592956 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.593364 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Traceback (most recent call last):
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.594243 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:51.595611 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.595928 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.597216 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:51.597456 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.597721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.597721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:51.597721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.597721 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-f60d98e6-2b31-45a4-a6cf-39a48dea4764 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.601839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:51.602360 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.602838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.602838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:51.602838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.602838 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:51.633325 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:51.633653 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:51.633762 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:51.633964 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:51.634570 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.634976 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:51.635305 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.636073 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.637394 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:51.637774 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.638154 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:51.639053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:51.639531 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:51.640140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:51.640140 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:56.137018 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:56.137462 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:56.137526 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:56.142704 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:56.183774 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.183971 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Traceback (most recent call last):
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.184670 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.185443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.186175 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-7bc2e5a3-8775-4830-95fd-306498318953 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:56.186906 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:56.187830 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.188935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:56.188935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.188935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.188935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:56.188935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.188935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:56.575196 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:56.575640 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:56.575851 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:56.577163 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.577582 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.577834 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.578865 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.580221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.580689 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:56.581039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.581039 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:56.581253 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.581682 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.582206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:56.582206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.582206 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:56.583437 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:56.584017 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:56.584161 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:56.584314 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:03:56.584884 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.585301 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.586308 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:56.587335 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.588018 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.588763 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:56.589221 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.589595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.589595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:56.589595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.589595 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:56.590086 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.590546 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.591053 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:56.591986 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:56.592542 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:56.592743 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:56.592955 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:56.593535 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.593901 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.594837 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:03:56.596287 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.596751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.597676 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:56.598065 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.598532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.598532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:56.598532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.598532 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.599432 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:56.599965 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.600507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.600507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:56.600507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.600507 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:03:56.634246 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:03:56.635613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:03:56.635613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:03:56.635613 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:03:56.636072 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.636431 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:03:56.636746 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.637659 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.639472 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:03:56.639963 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.640229 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:03:56.640773 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:03:56.641179 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:03:56.641558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:03:56.641558 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:04:01.137764 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:04:01.138718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:04:01.138718 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:04:01.140178 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:04:01.181941 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.182218 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Traceback (most recent call last):
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.182710 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.183758 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.184443 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-3acc6a69-659b-4dc3-929a-0f8abeedf660 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:04:01.185126 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:04:01.185935 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.187181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:04:01.187181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.187181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.187181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:04:01.187181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.187181 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:04:01.575898 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:04:01.576747 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:04:01.577043 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:04:01.578645 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-8dab20ec-1218-46be-8777-7aef7fbfc181, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.579082 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-530b037d-a47e-4f63-bd55-041139eeee2f, routes=[], if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.579450 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Traceback (most recent call last):
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.580716 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.582567 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.583449 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:04:01.584160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.584160 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-2140a6fa-371b-4e6d-8cd9-c51749a32441 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:04:01.584628 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.585832 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Setting fip-distributed flag in NB_Global to False {{(pid=107007) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 13 03:04:01.586842 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Traceback (most recent call last):
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.587893 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.588839 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.589750 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:04:01.590490 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-32ce9a00-4418-47dc-bf63-c87a828fb29c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:04:01.591241 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.591757 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.592534 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:04:01.593137 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:04:01.593351 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:04:01.593557 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:04:01.593772 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:04:01.594359 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.594751 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Traceback (most recent call last):
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.595549 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 13 03:04:01.596860 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.597428 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.598425 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:04:01.598884 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.599431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.599431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:04:01.599431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.599431 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [None req-9f25360a-4ba5-4c85-af53-3e45fb9e04e6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.600313 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:04:01.601054 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.601741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.601741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:04:01.601741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.601741 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics 
Apr 13 03:04:01.634919 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=107007) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 13 03:04:01.635205 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 13 03:04:01.635277 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=107007) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 13 03:04:01.635459 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=107007) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 13 03:04:01.636047 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.636362 np0000165409 neutron-ovn-maintenance-worker[107007]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=107007) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 13 03:04:01.636585 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]:     txn.results.put(txn.do_commit())
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]:                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]:     raise RuntimeError(msg)
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.637538 np0000165409 neutron-ovn-maintenance-worker[107007]: 
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.638525 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 13 03:04:01.639173 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics Traceback (most recent call last):
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     work()
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 13 03:04:01.639817 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     next(self.gen)
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     with self.create_transaction(
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     self.result = self.commit()
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise result.ex
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 13 03:04:01.640630 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 13 03:04:01.641801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 13 03:04:01.641801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 13 03:04:01.641801 np0000165409 neutron-ovn-maintenance-worker[107007]: ERROR futurist.periodics